[jira] [Resolved] (OAK-2679) Query engine: faster cost calculation
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller resolved OAK-2679. - Resolution: Fixed > Query engine: faster cost calculation > - > > Key: OAK-2679 > URL: https://issues.apache.org/jira/browse/OAK-2679 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, query >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Labels: performance > Fix For: 1.3.7 > > Attachments: 0001-OAK-2679-Reduce-execution-plan-overhead_0.2.patch, > OAK-2679-2.patch, OAK-2679.patch, executionplancache.patch > > > If there are many indexes, preparing a query can take a long time, in > relation to executing the query. > The query execution plans can be cached. The cache should be invalidated if > there are new indexes, or indexes are changed; a simple solution might be to > use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2679) Query engine: faster cost calculation
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14907686#comment-14907686 ] Thomas Mueller commented on OAK-2679: - Renamed the issue, as query plans are still not actually cached much. > Query engine: faster cost calculation > - > > Key: OAK-2679 > URL: https://issues.apache.org/jira/browse/OAK-2679 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, query >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Labels: performance > Fix For: 1.3.7 > > Attachments: 0001-OAK-2679-Reduce-execution-plan-overhead_0.2.patch, > OAK-2679-2.patch, OAK-2679.patch, executionplancache.patch > > > If there are many indexes, preparing a query can take a long time, in > relation to executing the query. > The query execution plans can be cached. The cache should be invalidated if > there are new indexes, or indexes are changed; a simple solution might be to > use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2679) Query engine: faster cost calculation
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-2679: Summary: Query engine: faster cost calculation (was: Query engine: cache execution plans) > Query engine: faster cost calculation > - > > Key: OAK-2679 > URL: https://issues.apache.org/jira/browse/OAK-2679 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, query >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Labels: performance > Fix For: 1.3.7 > > Attachments: 0001-OAK-2679-Reduce-execution-plan-overhead_0.2.patch, > OAK-2679-2.patch, OAK-2679.patch, executionplancache.patch > > > If there are many indexes, preparing a query can take a long time, in > relation to executing the query. > The query execution plans can be cached. The cache should be invalidated if > there are new indexes, or indexes are changed; a simple solution might be to > use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2679) Query engine: cache execution plans
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-2679: Attachment: OAK-2679-2.patch OAK-2679-2.patch > Query engine: cache execution plans > --- > > Key: OAK-2679 > URL: https://issues.apache.org/jira/browse/OAK-2679 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, query >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Labels: performance > Fix For: 1.3.7 > > Attachments: 0001-OAK-2679-Reduce-execution-plan-overhead_0.2.patch, > OAK-2679-2.patch, OAK-2679.patch, executionplancache.patch > > > If there are many indexes, preparing a query can take a long time, in > relation to executing the query. > The query execution plans can be cached. The cache should be invalidated if > there are new indexes, or indexes are changed; a simple solution might be to > use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3436) Prevent missing checkpoint due to unstable topology from causing complete reindexing
[
https://issues.apache.org/jira/browse/OAK-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14907623#comment-14907623
]
Chetan Mehrotra commented on OAK-3436:
--
Linking to OAK-2961 as similar warning about missing checkpoints were seen
there also. At that time we concluded that missing checkpoint can happen (like
case 1 here which eventually failed). The testcase can be checked to determine
if in some case after missing checkpoint index still progresses or not
> Prevent missing checkpoint due to unstable topology from causing complete
> reindexing
>
>
> Key: OAK-3436
> URL: https://issues.apache.org/jira/browse/OAK-3436
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.3.7, 1.2.7, 1.0.22
>
> Attachments: AsyncIndexUpdateClusterTest.java
>
>
> Async indexing logic relies on embedding application to ensure that async
> indexing job is run as a singleton in a cluster. For Sling based apps it
> depends on Sling Discovery support. At times it is being seen that if
> topology is not stable then different cluster nodes can consider them as
> leader and execute the async indexing job concurrently.
> This can cause problem as both cluster node might not see same repository
> state (due to write skew and eventual consistency) and might remove the
> checkpoint which other cluster node is still relying upon. For e.g. consider
> a 2 node cluster N1 and N2 where both are performing async indexing.
> # Base state - CP1 is the checkpoint for "async" job
> # N2 starts indexing and removes changes CP1 to CP2. For Mongo the
> checkpoints are saved in {{settings}} collection
> # N1 also decides to execute indexing but has yet not seen the latest
> repository state so still thinks that CP1 is the base checkpoint and tries to
> read it. However CP1 is already removed from {{settings}} and this makes N1
> think that checkpoint is missing and it decides to reindex everything!
> To avoid this topology must be stable but at Oak level we should still handle
> such a case and avoid doing a full reindexing. So we would need to have a
> {{MissingCheckpointStrategy}} similar to {{MissingIndexEditorStrategy}} as
> done in OAK-2203
> Possible approaches
> # A1 - Fail the indexing run if checkpoint is missing - Checkpoint being
> missing can have valid reason and invalid reason. Need to see what are valid
> scenarios where a checkpoint can go missing
> # A2 - When a checkpoint is created also store the creation time. When a
> checkpoint is found to be missing and its a *recent* checkpoint then fail the
> run. For e.g. we would fail the run till checkpoint found to be missing is
> less than an hour old (for just started take startup time into account)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3442) Intermittent IllegalMonitorStateException seen while releaseing IndexNode
[
https://issues.apache.org/jira/browse/OAK-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906901#comment-14906901
]
Alexander Klimetschek commented on OAK-3442:
The "Error occurred while obtaining InputStream for blobId" exception has been
happening for many days before the restart issue, always with the same blob
that is indeed missing on S3. As the index tried to index that binary again and
again. There was no indication of a broken index before the repo was restarted
(afaik).
I assume that exception is catched in the lucene index plugin and it doesn't
lead to a broken index - though that should be verified.
Otherwise something else, hidden, lead to the broken index.
> Intermittent IllegalMonitorStateException seen while releaseing IndexNode
> -
>
> Key: OAK-3442
> URL: https://issues.apache.org/jira/browse/OAK-3442
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.3.8
>
>
> At times following exception seen. On this system the index got corrupted
> because backing index files got deleted from the system and hence index is
> not accessible.
> {noformat}
> 21.09.2015 09:26:36.764 *ERROR* [FelixStartLevel]
> com.adobe.granite.repository.impl.SlingRepositoryManager start: Uncaught
> Throwable trying to access Repository, calling stopRepository()
> java.lang.IllegalMonitorStateException: attempt to unlock read lock, not
> locked by current thread
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.unmatchedUnlockException(ReentrantReadWriteLock.java:444)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryReleaseShared(ReentrantReadWriteLock.java:428)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.releaseShared(AbstractQueuedSynchronizer.java:1341)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.unlock(ReentrantReadWriteLock.java:881)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.release(IndexNode.java:121)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.getPlans(LucenePropertyIndex.java:212)
> at
> org.apache.jackrabbit.oak.query.QueryImpl.getBestSelectorExecutionPlan(QueryImpl.java:847)
> at
> org.apache.jackrabbit.oak.query.QueryImpl.getBestSelectorExecutionPlan(QueryImpl.java:793)
> at
> org.apache.jackrabbit.oak.query.ast.SelectorImpl.prepare(SelectorImpl.java:283)
> at
> org.apache.jackrabbit.oak.query.QueryImpl.prepare(QueryImpl.java:568)
> at
> org.apache.jackrabbit.oak.query.QueryEngineImpl.executeQuery(QueryEngineImpl.java:183)
> at
> org.apache.jackrabbit.oak.security.user.UserProvider.getAuthorizableByPrincipal(UserProvider.java:234)
> at
> org.apache.jackrabbit.oak.security.user.UserManagerImpl.getAuthorizable(UserManagerImpl.java:116)
> at
> org.apache.jackrabbit.oak.security.principal.PrincipalProviderImpl.getAuthorizable(PrincipalProviderImpl.java:140)
> at
> org.apache.jackrabbit.oak.security.principal.PrincipalProviderImpl.getPrincipal(PrincipalProviderImpl.java:69)
> at
> org.apache.jackrabbit.oak.spi.security.principal.CompositePrincipalProvider.getPrincipal(CompositePrincipalProvider.java:50)
> at
> org.apache.jackrabbit.oak.spi.security.principal.PrincipalManagerImpl.getPrincipal(PrincipalManagerImpl.java:47)
> at
> com.adobe.granite.repository.impl.SlingRepositoryManager.setupPermissions(SlingRepositoryManager.java:997)
> at
> com.adobe.granite.repository.impl.SlingRepositoryManager.createRepository(SlingRepositoryManager.java:420)
> at
> com.adobe.granite.repository.impl.SlingRepositoryManager.acquireRepository(SlingRepositoryManager.java:290)
> at
> org.apache.sling.jcr.base.AbstractSlingRepositoryManager.start(AbstractSlingRepositoryManager.java:304)
> at
> com.adobe.granite.repository.impl.SlingRepositoryManager.activate(SlingRepositoryManager.java:267)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at
> org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:222)
> at
> org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
> at
> org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:615)
> at
> org.apache.felix.scr.impl.help
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906643#comment-14906643
]
angela commented on OAK-3251:
-
[~tripod] created that test. so, from the top of my head i probably know less
that you, now that you already managed to reduce it a bit as you state above :-)
> speeding up the build time
> --
>
> Key: OAK-3251
> URL: https://issues.apache.org/jira/browse/OAK-3251
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Attachments: build-1441353866-times.log, build-1441353866.log,
> build-1441637422.log, build-1443006309.log,
> oak-build-for-unittests-times.log, oak-build-times.log, times-1441637422.log,
> times-1443006309.log
>
>
> Running the build with a {mvn clean install} takes a considerable amount of
> time: 15 minutes on my local.
> The top 10 slowest unit test are the following
> {noformat}
> 110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 54.012 sec
> org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
> 36.593 sec
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
> 35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
> 24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> 17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
> 16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
> 14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
> 11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> {noformat}
> Is there anything we could do to speed-up these?
> sorted times obtained with
> https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3436) Prevent missing checkpoint due to unstable topology from causing complete reindexing
[
https://issues.apache.org/jira/browse/OAK-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-3436:
-
Attachment: AsyncIndexUpdateClusterTest.java
Attaching [testcase|^AsyncIndexUpdateClusterTest.java] to reproduce the issue.
So far not able to do so!
So still need to figure out "Curious case of missing checkpoints"!
> Prevent missing checkpoint due to unstable topology from causing complete
> reindexing
>
>
> Key: OAK-3436
> URL: https://issues.apache.org/jira/browse/OAK-3436
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.3.7, 1.2.7, 1.0.22
>
> Attachments: AsyncIndexUpdateClusterTest.java
>
>
> Async indexing logic relies on embedding application to ensure that async
> indexing job is run as a singleton in a cluster. For Sling based apps it
> depends on Sling Discovery support. At times it is being seen that if
> topology is not stable then different cluster nodes can consider them as
> leader and execute the async indexing job concurrently.
> This can cause problem as both cluster node might not see same repository
> state (due to write skew and eventual consistency) and might remove the
> checkpoint which other cluster node is still relying upon. For e.g. consider
> a 2 node cluster N1 and N2 where both are performing async indexing.
> # Base state - CP1 is the checkpoint for "async" job
> # N2 starts indexing and removes changes CP1 to CP2. For Mongo the
> checkpoints are saved in {{settings}} collection
> # N1 also decides to execute indexing but has yet not seen the latest
> repository state so still thinks that CP1 is the base checkpoint and tries to
> read it. However CP1 is already removed from {{settings}} and this makes N1
> think that checkpoint is missing and it decides to reindex everything!
> To avoid this topology must be stable but at Oak level we should still handle
> such a case and avoid doing a full reindexing. So we would need to have a
> {{MissingCheckpointStrategy}} similar to {{MissingIndexEditorStrategy}} as
> done in OAK-2203
> Possible approaches
> # A1 - Fail the indexing run if checkpoint is missing - Checkpoint being
> missing can have valid reason and invalid reason. Need to see what are valid
> scenarios where a checkpoint can go missing
> # A2 - When a checkpoint is created also store the creation time. When a
> checkpoint is found to be missing and its a *recent* checkpoint then fail the
> run. For e.g. we would fail the run till checkpoint found to be missing is
> less than an hour old (for just started take startup time into account)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3436) Prevent missing checkpoint due to unstable topology from causing complete reindexing
[
https://issues.apache.org/jira/browse/OAK-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906506#comment-14906506
]
Chetan Mehrotra commented on OAK-3436:
--
After being pointed out by Alex above I tried to recreate the scenario and so
far not able to get it going. The lease check does prevent concurrent
execution. From customer logs following can be made out
h3. Cluster Node N2
*run-n2-1*
{noformat}
16.09.2015 15:29:57.292 *WARN* [pool-5-thread-1]
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate Failed to retrieve
previously indexed checkpoint r14fd6c63cbf-0-3; re-running the initial async
index update
16.09.2015 15:29:57.314 *INFO* [pool-5-thread-1]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Found a new index node
[authorizables]. Reindexing is requested
16.09.2015 15:30:14.009 *INFO* [pool-5-thread-1]
org.apache.jackrabbit.oak.plugins.index.IndexUpdate Found a new index node
[counter]. Reindexing is requested
16.09.2015 15:39:29.262 *INFO* [pool-5-thread-1]
org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore query timed
out after 3000 milliseconds and will be retried without lock
[4:/oak:index/uuid/:index/, 4:/oak:index/uuid/:index0, _modified, 1442417375,
2147483647]
16.09.2015 15:40:47.235 *INFO* [pool-5-thread-1]
org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore query timed
out after 3000 milliseconds and will be retried without lock
[4:/oak:index/uuid/:index/, 4:/oak:index/uuid/:index0, _modified, 1442417375,
2147483647]
16.09.2015 15:41:59.617 *ERROR* [pool-5-thread-1]
org.apache.sling.commons.scheduler.impl.QuartzScheduler Exception during job
execution of org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate@74561c24
: operation exceeded time limit
com.mongodb.MongoExecutionTimeoutException: operation exceeded time limit
at
com.mongodb.QueryResultIterator.throwOnQueryFailure(QueryResultIterator.java:244)
at com.mongodb.QueryResultIterator.init(QueryResultIterator.java:224)
at
com.mongodb.QueryResultIterator.initFromQueryResponse(QueryResultIterator.java:184)
at com.mongodb.QueryResultIterator.(QueryResultIterator.java:62)
at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:86)
at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:66)
at com.mongodb.DBCursor._check(DBCursor.java:498)
at com.mongodb.DBCursor._hasNext(DBCursor.java:621)
at com.mongodb.DBCursor.hasNext(DBCursor.java:657)
at
org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.queryInternal(MongoDocumentStore.java:647)
at
org.apache.jackrabbit.oak.plugins.document.mongo.MongoDocumentStore.query(MongoDocumentStore.java:578)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.diffManyChildren(DocumentNodeStore.java:2237)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.diffImpl(DocumentNodeStore.java:2205)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.access$400(DocumentNodeStore.java:122)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore$10.call(DocumentNodeStore.java:1395)
at
org.apache.jackrabbit.oak.plugins.document.MemoryDiffCache.getChanges(MemoryDiffCache.java:74)
at
org.apache.jackrabbit.oak.plugins.document.TieredDiffCache.getChanges(TieredDiffCache.java:50)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.compare(DocumentNodeStore.java:1390)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeState.compareAgainstBaseState(DocumentNodeState.java:343)
at
org.apache.jackrabbit.oak.plugins.document.CommitDiff.childNodeChanged(CommitDiff.java:96)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.dispatch(DocumentNodeStore.java:2091)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.compare(DocumentNodeStore.java:1390)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeState.compareAgainstBaseState(DocumentNodeState.java:343)
at
org.apache.jackrabbit.oak.plugins.document.CommitDiff.childNodeChanged(CommitDiff.java:96)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState$1.childNodeChanged(ModifiedNodeState.java:429)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.dispatch(DocumentNodeStore.java:2091)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.compare(DocumentNodeStore.java:1390)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeState.compareAgainstBaseState(DocumentNodeState.java:343)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:405)
at
org.apache.jackrabbit.oak.plugins.document.CommitDiff.childNodeChanged(CommitDiff.java:96)
at
org.apache.jac
[jira] [Commented] (OAK-3417) oak-run OakFixture might set incorrect clusterIDs
[
https://issues.apache.org/jira/browse/OAK-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906462#comment-14906462
]
Julian Reschke commented on OAK-3417:
-
Further investigation shows that there is no code in oak-run that actually uses
clustered setups. Fix or remove?
> oak-run OakFixture might set incorrect clusterIDs
> -
>
> Key: OAK-3417
> URL: https://issues.apache.org/jira/browse/OAK-3417
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: run
>Reporter: Julian Reschke
>Priority: Minor
>
> Seen in OakFixture.java:
> {code}
> @Override
> public Oak[] setUpCluster(int n) throws Exception {
> Oak[] cluster = new Oak[n];
> kernels = new DocumentMK[cluster.length];
> for (int i = 0; i < cluster.length; i++) {
> MongoConnection mongo = new MongoConnection(uri);
> DocumentMK.Builder mkBuilder = new DocumentMK.Builder().
> setMongoDB(mongo.getDB()).
> memoryCacheSize(cacheSize).
> setPersistentCache("target/persistentCache,time").
> setClusterId(i).
> setLogging(false);
> setupBlobStore(mkBuilder);
> kernels[i] = mkBuilder.open();
> cluster[i] = newOak(kernels[i].getNodeStore());
> }
> return cluster;
> }
> {code}
> ...however, {{setClusterId(0)}} is special (it generates it automatically}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2679) Query engine: cache execution plans
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906421#comment-14906421 ] Thomas Mueller commented on OAK-2679: - I will change the minimum costs per index type slightly: * 0.0 for uuid * 0.0 for traversal * 1.0 for reference * 2.0 for property * 2.0 for nodetype * 2.1 for lucene property * 2.2 for lucene (old) * 2.3 for solr * 3.0 for ordered (deprecated) > Query engine: cache execution plans > --- > > Key: OAK-2679 > URL: https://issues.apache.org/jira/browse/OAK-2679 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, query >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Labels: performance > Fix For: 1.3.7 > > Attachments: 0001-OAK-2679-Reduce-execution-plan-overhead_0.2.patch, > OAK-2679.patch, executionplancache.patch > > > If there are many indexes, preparing a query can take a long time, in > relation to executing the query. > The query execution plans can be cached. The cache should be invalidated if > there are new indexes, or indexes are changed; a simple solution might be to > use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-3420) DocumentNodeStoreService fails to restart DocumentNodeStore
[ https://issues.apache.org/jira/browse/OAK-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-3420. - Resolution: Fixed Fix Version/s: 1.0.22 trunk: http://svn.apache.org/r1705055 1.2: http://svn.apache.org/r1705065 1.0: http://svn.apache.org/r1705069 > DocumentNodeStoreService fails to restart DocumentNodeStore > --- > > Key: OAK-3420 > URL: https://issues.apache.org/jira/browse/OAK-3420 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, mongomk, rdbmk >Affects Versions: 1.0.21, 1.3.6, 1.2.6 >Reporter: Julian Reschke >Assignee: Julian Reschke > Labels: resilience > Fix For: 1.3.7, 1.2.7, 1.0.22 > > Attachments: DocumentNodeStoreConfigTest.groovy > > > Scenario (test case will follow): > 1) service configured for RDBDocumentStore with DataSources > 2) gets started > 3) DataSource unregisters, DocumentNodeStore gets shut down > 4) DataSource is registered again > 5) restart of DocumentNodeStore fails with NPE because at least one required > component (executor?) now is null > We need to decide whether this is a scenario that should be supported or not. > If yes, we need to fix it. If no, we need to make the code more robust and > improve diagnostics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3420) DocumentNodeStoreService fails to restart DocumentNodeStore
[ https://issues.apache.org/jira/browse/OAK-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3420: Fix Version/s: 1.2.7 > DocumentNodeStoreService fails to restart DocumentNodeStore > --- > > Key: OAK-3420 > URL: https://issues.apache.org/jira/browse/OAK-3420 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, mongomk, rdbmk >Affects Versions: 1.0.21, 1.3.6, 1.2.6 >Reporter: Julian Reschke >Assignee: Julian Reschke > Labels: resilience > Fix For: 1.3.7, 1.2.7 > > Attachments: DocumentNodeStoreConfigTest.groovy > > > Scenario (test case will follow): > 1) service configured for RDBDocumentStore with DataSources > 2) gets started > 3) DataSource unregisters, DocumentNodeStore gets shut down > 4) DataSource is registered again > 5) restart of DocumentNodeStore fails with NPE because at least one required > component (executor?) now is null > We need to decide whether this is a scenario that should be supported or not. > If yes, we need to fix it. If no, we need to make the code more robust and > improve diagnostics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906365#comment-14906365
]
Davide Giannella commented on OAK-3251:
---
I think then we can consider this issue as resolved. We did what was a quick
win to speed up the things and the build moved 3 minutes faster in my local.
This can work as well as analysis for identifying the slowest tests.
If no one will object I will mark it resolved on Monday.
> speeding up the build time
> --
>
> Key: OAK-3251
> URL: https://issues.apache.org/jira/browse/OAK-3251
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Attachments: build-1441353866-times.log, build-1441353866.log,
> build-1441637422.log, build-1443006309.log,
> oak-build-for-unittests-times.log, oak-build-times.log, times-1441637422.log,
> times-1443006309.log
>
>
> Running the build with a {mvn clean install} takes a considerable amount of
> time: 15 minutes on my local.
> The top 10 slowest unit test are the following
> {noformat}
> 110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 54.012 sec
> org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
> 36.593 sec
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
> 35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
> 24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> 17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
> 16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
> 14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
> 11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> {noformat}
> Is there anything we could do to speed-up these?
> sorted times obtained with
> https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-1312) Bundle nodes into a document
[ https://issues.apache.org/jira/browse/OAK-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906361#comment-14906361 ] Michael Dürig commented on OAK-1312: Digging out an [old conversation|http://jackrabbit.markmail.org/message/mfe3worjyfvptdod?q=node+type] here: bq. Node types may give hints here. As long as they are not recursive (i.e. nt:hierarchy) node types usually define "things that belong together". > Bundle nodes into a document > > > Key: OAK-1312 > URL: https://issues.apache.org/jira/browse/OAK-1312 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, mongomk >Reporter: Marcel Reutegger > > For very fine grained content with many nodes and only few properties per > node it would be more efficient to bundle multiple nodes into a single > MongoDB document. Mostly reading would benefit because there are less > roundtrips to the backend. At the same time storage footprint would be lower > because metadata overhead is per document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3447) Parallelize recursive usage of compareAgainstBaseState
[ https://issues.apache.org/jira/browse/OAK-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906360#comment-14906360 ] Thomas Mueller commented on OAK-3447: - It would be good to know the average and medium number of child nodes processed. > Parallelize recursive usage of compareAgainstBaseState > -- > > Key: OAK-3447 > URL: https://issues.apache.org/jira/browse/OAK-3447 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Affects Versions: 1.3.6 > Environment: All, especially document >Reporter: Joel Richard > Labels: performance > > In order to improve the performance of compareAgainstBaseState, it would help > to parallelize the recursive usage of compareAgainstBaseState. The idea is > that each sub tree which has a different revision would then be processed in > parallel (although it would probably suffice to only fork the process when > the nodes are not cached). This should significantly reduce the time which is > lost while waiting for an external database assumed that there are at least > two changes between the base revisions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2808) Active deletion of 'deleted' Lucene index files from DataStore without relying on full scale Blob GC
[ https://issues.apache.org/jira/browse/OAK-2808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906356#comment-14906356 ] Thomas Mueller commented on OAK-2808: - http://svn.apache.org/r1705054 (trunk), partial implementation. Remarks: * Disabled by default (DEFAULT_ACTIVE_DELETE = -1). * Delete of entries is not yet implemented; this would have to be done in oak-core I guess. * To enable, set the property "activeDelete" to, for example, 3600 (1 hour), for an index of type "lucene" * For manual testing, I removed the "async" = "async" flag from the index - this will create garbage quickly. * A new child node ":trash" is created, with child nodes "run_1", "run_2",... and a property "index" for the next id. * For each file, a new property "uniqueKey" (16 bytes) is created, and that key is appended to the file (ignored when reading) The block size per binary increased by 16 bytes. I wonder if, for MongoDB, it would be better to use 1024 bytes less, as we do for the MongoBlobStore, because MongoDB rounds up the space allocated for a record to the next power of two (there is an overhead per record, let's assume it is 1 KB at most) The missing "delete trash" feature would need to periodically (and asynchronously) read the first "run_.." entries, and delete the binaries if needed. It would probably have to maintain a "deleteIndex", similar to the "index" property used to create new entries. > Active deletion of 'deleted' Lucene index files from DataStore without > relying on full scale Blob GC > > > Key: OAK-2808 > URL: https://issues.apache.org/jira/browse/OAK-2808 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: lucene >Reporter: Chetan Mehrotra >Assignee: Thomas Mueller > Labels: datastore, performance > Fix For: 1.3.7 > > Attachments: OAK-2808-1.patch, copyonread-stats.png > > > With storing of Lucene index files within DataStore our usage pattern > of DataStore has changed between JR2 and Oak. > With JR2 the writes were mostly application based i.e. if application > stores a pdf/image file then that would be stored in DataStore. JR2 by > default would not write stuff to DataStore. Further in deployment > where large number of binary content is present then systems tend to > share the DataStore to avoid duplication of storage. In such cases > running Blob GC is a non trivial task as it involves a manual step and > coordination across multiple deployments. Due to this systems tend to > delay frequency of GC > Now with Oak apart from application the Oak system itself *actively* > uses the DataStore to store the index files for Lucene and there the > churn might be much higher i.e. frequency of creation and deletion of > index file is lot higher. This would accelerate the rate of garbage > generation and thus put lot more pressure on the DataStore storage > requirements. > Discussion thread http://markmail.org/thread/iybd3eq2bh372zrl -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3447) Parallelize recursive usage of compareAgainstBaseState
Joel Richard created OAK-3447: - Summary: Parallelize recursive usage of compareAgainstBaseState Key: OAK-3447 URL: https://issues.apache.org/jira/browse/OAK-3447 Project: Jackrabbit Oak Issue Type: Improvement Components: core Affects Versions: 1.3.6 Environment: All, especially document Reporter: Joel Richard In order to improve the performance of compareAgainstBaseState, it would help to parallelize the recursive usage of compareAgainstBaseState. The idea is that each sub tree which has a different revision would then be processed in parallel (although it would probably suffice to only fork the process when the nodes are not cached). This should significantly reduce the time which is lost while waiting for an external database assumed that there are at least two changes between the base revisions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3420) DocumentNodeStoreService fails to restart DocumentNodeStore
[ https://issues.apache.org/jira/browse/OAK-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906341#comment-14906341 ] Julian Reschke commented on OAK-3420: - for now applying a simple fix that just avoids any attempts of restarting; this at least avoids the NPE we've had before (which may indicate an attempt to start a DocumentNodeStore that may not get disposed again) > DocumentNodeStoreService fails to restart DocumentNodeStore > --- > > Key: OAK-3420 > URL: https://issues.apache.org/jira/browse/OAK-3420 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, mongomk, rdbmk >Affects Versions: 1.0.21, 1.3.6, 1.2.6 >Reporter: Julian Reschke >Assignee: Julian Reschke > Labels: resilience > Fix For: 1.3.7 > > Attachments: DocumentNodeStoreConfigTest.groovy > > > Scenario (test case will follow): > 1) service configured for RDBDocumentStore with DataSources > 2) gets started > 3) DataSource unregisters, DocumentNodeStore gets shut down > 4) DataSource is registered again > 5) restart of DocumentNodeStore fails with NPE because at least one required > component (executor?) now is null > We need to decide whether this is a scenario that should be supported or not. > If yes, we need to fix it. If no, we need to make the code more robust and > improve diagnostics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-3420) DocumentNodeStoreService fails to restart DocumentNodeStore
[ https://issues.apache.org/jira/browse/OAK-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke reassigned OAK-3420: --- Assignee: Julian Reschke > DocumentNodeStoreService fails to restart DocumentNodeStore > --- > > Key: OAK-3420 > URL: https://issues.apache.org/jira/browse/OAK-3420 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, mongomk, rdbmk >Affects Versions: 1.0.21, 1.3.6, 1.2.6 >Reporter: Julian Reschke >Assignee: Julian Reschke > Labels: resilience > Fix For: 1.3.7 > > Attachments: DocumentNodeStoreConfigTest.groovy > > > Scenario (test case will follow): > 1) service configured for RDBDocumentStore with DataSources > 2) gets started > 3) DataSource unregisters, DocumentNodeStore gets shut down > 4) DataSource is registered again > 5) restart of DocumentNodeStore fails with NPE because at least one required > component (executor?) now is null > We need to decide whether this is a scenario that should be supported or not. > If yes, we need to fix it. If no, we need to make the code more robust and > improve diagnostics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3420) DocumentNodeStoreService fails to restart DocumentNodeStore
[ https://issues.apache.org/jira/browse/OAK-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3420: Fix Version/s: 1.3.7 > DocumentNodeStoreService fails to restart DocumentNodeStore > --- > > Key: OAK-3420 > URL: https://issues.apache.org/jira/browse/OAK-3420 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, mongomk, rdbmk >Affects Versions: 1.0.21, 1.3.6, 1.2.6 >Reporter: Julian Reschke > Labels: resilience > Fix For: 1.3.7 > > Attachments: DocumentNodeStoreConfigTest.groovy > > > Scenario (test case will follow): > 1) service configured for RDBDocumentStore with DataSources > 2) gets started > 3) DataSource unregisters, DocumentNodeStore gets shut down > 4) DataSource is registered again > 5) restart of DocumentNodeStore fails with NPE because at least one required > component (executor?) now is null > We need to decide whether this is a scenario that should be supported or not. > If yes, we need to fix it. If no, we need to make the code more robust and > improve diagnostics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-1312) Bundle nodes into a document
[ https://issues.apache.org/jira/browse/OAK-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906328#comment-14906328 ] Joel Richard commented on OAK-1312: --- One approach to decide which nodes should be saved together in a document could be to leverage the node type and mixin information: If the type or mixins of a node allow or require certain child nodes, then these child nodes would be stored together with the parent node in a single document. Since a child node isn't read before it's parent anyway, nodes could still be retrieved in the old way. Example: Since a node with access control needs to have the rep:AccessControllable mixin and the mixin suggests that a node with this mixin might have the rep:policy child node, these two nodes would be stored together. > Bundle nodes into a document > > > Key: OAK-1312 > URL: https://issues.apache.org/jira/browse/OAK-1312 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, mongomk >Reporter: Marcel Reutegger > > For very fine grained content with many nodes and only few properties per > node it would be more efficient to bundle multiple nodes into a single > MongoDB document. Mostly reading would benefit because there are less > roundtrips to the backend. At the same time storage footprint would be lower > because metadata overhead is per document. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3420) DocumentNodeStoreService fails to restart DocumentNodeStore
[ https://issues.apache.org/jira/browse/OAK-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3420: Affects Version/s: 1.0.21 1.3.6 1.2.6 > DocumentNodeStoreService fails to restart DocumentNodeStore > --- > > Key: OAK-3420 > URL: https://issues.apache.org/jira/browse/OAK-3420 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, mongomk, rdbmk >Affects Versions: 1.0.21, 1.3.6, 1.2.6 >Reporter: Julian Reschke > Labels: resilience > Attachments: DocumentNodeStoreConfigTest.groovy > > > Scenario (test case will follow): > 1) service configured for RDBDocumentStore with DataSources > 2) gets started > 3) DataSource unregisters, DocumentNodeStore gets shut down > 4) DataSource is registered again > 5) restart of DocumentNodeStore fails with NPE because at least one required > component (executor?) now is null > We need to decide whether this is a scenario that should be supported or not. > If yes, we need to fix it. If no, we need to make the code more robust and > improve diagnostics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3420) DocumentNodeStoreService fails to restart DocumentNodeStore
[ https://issues.apache.org/jira/browse/OAK-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3420: Component/s: rdbmk mongomk > DocumentNodeStoreService fails to restart DocumentNodeStore > --- > > Key: OAK-3420 > URL: https://issues.apache.org/jira/browse/OAK-3420 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, mongomk, rdbmk >Affects Versions: 1.0.21, 1.3.6, 1.2.6 >Reporter: Julian Reschke > Labels: resilience > Attachments: DocumentNodeStoreConfigTest.groovy > > > Scenario (test case will follow): > 1) service configured for RDBDocumentStore with DataSources > 2) gets started > 3) DataSource unregisters, DocumentNodeStore gets shut down > 4) DataSource is registered again > 5) restart of DocumentNodeStore fails with NPE because at least one required > component (executor?) now is null > We need to decide whether this is a scenario that should be supported or not. > If yes, we need to fix it. If no, we need to make the code more robust and > improve diagnostics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-3446) RDBDocumentStore: update PostgresQL and MySQL JDBC drivers
[ https://issues.apache.org/jira/browse/OAK-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-3446. - Resolution: Fixed Fix Version/s: 1.0.22 trunk: http://svn.apache.org/r1705043 1.2: http://svn.apache.org/r1705044 1.0: http://svn.apache.org/r1705045 > RDBDocumentStore: update PostgresQL and MySQL JDBC drivers > -- > > Key: OAK-3446 > URL: https://issues.apache.org/jira/browse/OAK-3446 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.21, 1.3.6, 1.2.6 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Fix For: 1.3.7, 1.2.7, 1.0.22 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3446) RDBDocumentStore: update PostgresQL and MySQL JDBC drivers
[ https://issues.apache.org/jira/browse/OAK-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3446: Fix Version/s: 1.2.7 > RDBDocumentStore: update PostgresQL and MySQL JDBC drivers > -- > > Key: OAK-3446 > URL: https://issues.apache.org/jira/browse/OAK-3446 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.21, 1.3.6, 1.2.6 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Fix For: 1.3.7, 1.2.7 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3446) RDBDocumentStore: update PostgresQL and MySQL JDBC drivers
[ https://issues.apache.org/jira/browse/OAK-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3446: Fix Version/s: 1.3.7 > RDBDocumentStore: update PostgresQL and MySQL JDBC drivers > -- > > Key: OAK-3446 > URL: https://issues.apache.org/jira/browse/OAK-3446 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.0.21, 1.3.6, 1.2.6 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Trivial > Fix For: 1.3.7 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3446) RDBDocumentStore: update PostgresQL and MySQL JDBC drivers
Julian Reschke created OAK-3446: --- Summary: RDBDocumentStore: update PostgresQL and MySQL JDBC drivers Key: OAK-3446 URL: https://issues.apache.org/jira/browse/OAK-3446 Project: Jackrabbit Oak Issue Type: Technical task Components: rdbmk Affects Versions: 1.2.6, 1.3.6, 1.0.21 Reporter: Julian Reschke Assignee: Julian Reschke Priority: Trivial -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-1591) org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT fails
[ https://issues.apache.org/jira/browse/OAK-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906261#comment-14906261 ] Marcel Reutegger commented on OAK-1591: --- Yes, please. This issue is closed and cannot be re-opened. > org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT fails > -- > > Key: OAK-1591 > URL: https://issues.apache.org/jira/browse/OAK-1591 > Project: Jackrabbit Oak > Issue Type: Task > Components: mongomk >Reporter: Julian Reschke >Assignee: Chetan Mehrotra >Priority: Minor > Labels: CI > Fix For: 1.3.6 > > > Fails frequently on my W7 desktop: > testCacheInvalidationHierarchicalNotExist(org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT) > Time elapsed: 0.04 sec <<< FAILURE! > java.lang.AssertionError > at org.junit.Assert.fail(Assert.java:92) > at org.junit.Assert.assertTrue(Assert.java:43) > at org.junit.Assert.assertTrue(Assert.java:54) > at > org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT.testCacheInvalidationHierarchicalNotExist(CacheInvalidationIT.java:171) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-1591) org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT fails
[
https://issues.apache.org/jira/browse/OAK-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906246#comment-14906246
]
Julian Reschke edited comment on OAK-1591 at 9/24/15 11:52 AM:
---
It's now failing again reliably; maybe something changed, maybe I wasn't
testing with mongodb active. Stacktrace:
{code}
testCacheInvalidationHierarchical(org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT)
Time elapsed: 0.031 sec <<< FAILURE!
java.lang.AssertionError: expected:<2> but was:<0>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at
org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT.testCacheInvalidationHierarchical(CacheInvalidationIT.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
{code}
[~mreutegg]: should I reopen or create a new ticket?
was (Author: reschke):
It's now failing again reliably; maybe something changed, maybe I wasn't
testing with mongodb active. Stacktrace:
{code}
testCacheInvalidationHierarchical(org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT)
Time elapsed: 0.031 sec <<< FAILURE!
java.lang.AssertionError: expected:<2> but was:<0>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at
org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT.testCacheInvalidationHierarchical(CacheInvalidationIT.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
{code}
> org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT fails
> --
>
> Key: OAK-1591
> URL: https://issues.apache.org/jira/browse/OAK-1591
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: mongomk
>Reporter: Julian Reschke
>Assignee: Chetan Mehrotra
>Priority: Minor
> Labels: CI
> Fix For: 1.3.6
>
>
> Fails frequently on my W7 desktop:
> testCacheInvalidationHierarchicalNotExist(org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT)
> Time elapsed: 0.04 sec <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:92)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertTrue(Assert.java:54)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT.testCacheInvalidationHierarchicalNotExist(CacheInvalidationIT.java:171)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-1591) org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT fails
[
https://issues.apache.org/jira/browse/OAK-1591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906246#comment-14906246
]
Julian Reschke commented on OAK-1591:
-
It's now failing again reliably; maybe something changed, maybe I wasn't
testing with mongodb active. Stacktrace:
{code}
testCacheInvalidationHierarchical(org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT)
Time elapsed: 0.031 sec <<< FAILURE!
java.lang.AssertionError: expected:<2> but was:<0>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at
org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT.testCacheInvalidationHierarchical(CacheInvalidationIT.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
{code}
> org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT fails
> --
>
> Key: OAK-1591
> URL: https://issues.apache.org/jira/browse/OAK-1591
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: mongomk
>Reporter: Julian Reschke
>Assignee: Chetan Mehrotra
>Priority: Minor
> Labels: CI
> Fix For: 1.3.6
>
>
> Fails frequently on my W7 desktop:
> testCacheInvalidationHierarchicalNotExist(org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT)
> Time elapsed: 0.04 sec <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:92)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertTrue(Assert.java:54)
> at
> org.apache.jackrabbit.oak.plugins.document.mongo.CacheInvalidationIT.testCacheInvalidationHierarchicalNotExist(CacheInvalidationIT.java:171)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3434) Revert backwards-incompatible changes to SecurityProviderImpl
[
https://issues.apache.org/jira/browse/OAK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906240#comment-14906240
]
Francesco Mari commented on OAK-3434:
-
Backported to 1.0 in r1705037.
> Revert backwards-incompatible changes to SecurityProviderImpl
> -
>
> Key: OAK-3434
> URL: https://issues.apache.org/jira/browse/OAK-3434
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: security
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7, 1.0.22
>
> Attachments: OAK-3434-01.patch
>
>
> OAK-3201 introduced some backwards-incompatible changes to
> {{SecurityProviderImpl}}. It should be investigated if those changes can be
> reverted to maintain the backwards compatibility of the
> {{org.apache.jackrabbit.oak.security}} package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3441) SecurityProviderImpl should not be an OSGi component
[ https://issues.apache.org/jira/browse/OAK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francesco Mari updated OAK-3441: Fix Version/s: 1.0.22 > SecurityProviderImpl should not be an OSGi component > > > Key: OAK-3441 > URL: https://issues.apache.org/jira/browse/OAK-3441 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: security >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.3.7, 1.2.7, 1.0.22 > > > With the introduction of SecurityProviderRegistration in OAK-3201, > SecurityProviderImpl doesn't need to be registered as an OSGi component > anymore. SecurityProviderImpl should be turned into a plain class in a > backwards compatible way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3441) SecurityProviderImpl should not be an OSGi component
[ https://issues.apache.org/jira/browse/OAK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906241#comment-14906241 ] Francesco Mari commented on OAK-3441: - Backported to 1.0 in r1705037. > SecurityProviderImpl should not be an OSGi component > > > Key: OAK-3441 > URL: https://issues.apache.org/jira/browse/OAK-3441 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: security >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.3.7, 1.2.7, 1.0.22 > > > With the introduction of SecurityProviderRegistration in OAK-3201, > SecurityProviderImpl doesn't need to be registered as an OSGi component > anymore. SecurityProviderImpl should be turned into a plain class in a > backwards compatible way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3431) SecurityProviderRegistration should not be part of an exported package
[ https://issues.apache.org/jira/browse/OAK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906239#comment-14906239 ] Francesco Mari commented on OAK-3431: - Backported to 1.0 in r1705037. > SecurityProviderRegistration should not be part of an exported package > -- > > Key: OAK-3431 > URL: https://issues.apache.org/jira/browse/OAK-3431 > Project: Jackrabbit Oak > Issue Type: Bug > Components: security >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.3.7, 1.2.7, 1.0.22 > > > The SecurityProviderRegistration component should not be part of an exported > package. This component should be treated like an implementation detail of > the oak-core bundle, and no client code should directly depend on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3423) RandomAuthorizableNodeName should not be part of the default configuration of SecurityProviderImpl
[
https://issues.apache.org/jira/browse/OAK-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-3423:
Fix Version/s: 1.0.22
> RandomAuthorizableNodeName should not be part of the default configuration of
> SecurityProviderImpl
> --
>
> Key: OAK-3423
> URL: https://issues.apache.org/jira/browse/OAK-3423
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: security
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7, 1.0.22
>
>
> OAK-3201 identified {{RandomAuthorizableNodeName}} as the default choice for
> {{SecurityProviderImpl}}. This choice is wrong and should be reverted. In
> particular, {{RandomAuthorizableNodeName}} should switch back to a required
> configuration policy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3434) Revert backwards-incompatible changes to SecurityProviderImpl
[
https://issues.apache.org/jira/browse/OAK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-3434:
Fix Version/s: 1.0.22
> Revert backwards-incompatible changes to SecurityProviderImpl
> -
>
> Key: OAK-3434
> URL: https://issues.apache.org/jira/browse/OAK-3434
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: security
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7, 1.0.22
>
> Attachments: OAK-3434-01.patch
>
>
> OAK-3201 introduced some backwards-incompatible changes to
> {{SecurityProviderImpl}}. It should be investigated if those changes can be
> reverted to maintain the backwards compatibility of the
> {{org.apache.jackrabbit.oak.security}} package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3431) SecurityProviderRegistration should not be part of an exported package
[ https://issues.apache.org/jira/browse/OAK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francesco Mari updated OAK-3431: Fix Version/s: 1.0.22 > SecurityProviderRegistration should not be part of an exported package > -- > > Key: OAK-3431 > URL: https://issues.apache.org/jira/browse/OAK-3431 > Project: Jackrabbit Oak > Issue Type: Bug > Components: security >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.3.7, 1.2.7, 1.0.22 > > > The SecurityProviderRegistration component should not be part of an exported > package. This component should be treated like an implementation detail of > the oak-core bundle, and no client code should directly depend on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3201) Use static references in SecurityProviderImpl for composite services
[
https://issues.apache.org/jira/browse/OAK-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906236#comment-14906236
]
Francesco Mari commented on OAK-3201:
-
Backported to 1.0 in r1705037.
> Use static references in SecurityProviderImpl for composite services
>
>
> Key: OAK-3201
> URL: https://issues.apache.org/jira/browse/OAK-3201
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7, 1.0.22
>
> Attachments: OAK-3201-01.patch, OAK-3201-02.patch, OAK-3201-03.patch,
> OAK-3201-04.patch, OAK-3201-05.patch, OAK-3201-06.patch, OAK-3201-07.patch,
> OAK-3201-08.patch, OAK-3201-09.patch, mbean-test.log
>
>
> {{SecurityProviderImpl}} has dynamic references to many other services, like
> {{RestrictionProvider}}, that represent the configuration of this component.
> Being these services dynamic, the OSGi runtime has no clear dependency
> relationship between the {{SecurityProviderImpl}} and the required services.
> Thus, it may happen that an instance of {{SecurityProviderImpl}} is published
> before the services it requires are started, creating a window where the
> {{SecurityProviderimpl}} is operating differently from the way it's
> configured.
> I suggest to turn the dynamic references in {{SecurityProviderImpl}} to
> static ones to improve the consistency of the implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3201) Use static references in SecurityProviderImpl for composite services
[
https://issues.apache.org/jira/browse/OAK-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-3201:
Fix Version/s: 1.0.22
> Use static references in SecurityProviderImpl for composite services
>
>
> Key: OAK-3201
> URL: https://issues.apache.org/jira/browse/OAK-3201
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7, 1.0.22
>
> Attachments: OAK-3201-01.patch, OAK-3201-02.patch, OAK-3201-03.patch,
> OAK-3201-04.patch, OAK-3201-05.patch, OAK-3201-06.patch, OAK-3201-07.patch,
> OAK-3201-08.patch, OAK-3201-09.patch, mbean-test.log
>
>
> {{SecurityProviderImpl}} has dynamic references to many other services, like
> {{RestrictionProvider}}, that represent the configuration of this component.
> Being these services dynamic, the OSGi runtime has no clear dependency
> relationship between the {{SecurityProviderImpl}} and the required services.
> Thus, it may happen that an instance of {{SecurityProviderImpl}} is published
> before the services it requires are started, creating a window where the
> {{SecurityProviderimpl}} is operating differently from the way it's
> configured.
> I suggest to turn the dynamic references in {{SecurityProviderImpl}} to
> static ones to improve the consistency of the implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3423) RandomAuthorizableNodeName should not be part of the default configuration of SecurityProviderImpl
[
https://issues.apache.org/jira/browse/OAK-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906237#comment-14906237
]
Francesco Mari commented on OAK-3423:
-
Backported to 1.0 in r1705037.
> RandomAuthorizableNodeName should not be part of the default configuration of
> SecurityProviderImpl
> --
>
> Key: OAK-3423
> URL: https://issues.apache.org/jira/browse/OAK-3423
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: security
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7
>
>
> OAK-3201 identified {{RandomAuthorizableNodeName}} as the default choice for
> {{SecurityProviderImpl}}. This choice is wrong and should be reverted. In
> particular, {{RandomAuthorizableNodeName}} should switch back to a required
> configuration policy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-3445) RDBDocumentStore: when generating SQL for queries, leave out unneeded constraints
[
https://issues.apache.org/jira/browse/OAK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke resolved OAK-3445.
-
Resolution: Fixed
trunk: http://svn.apache.org/r1705027
1.2: http://svn.apache.org/r1705030
1.0: http://svn.apache.org/r1705034
> RDBDocumentStore: when generating SQL for queries, leave out unneeded
> constraints
> -
>
> Key: OAK-3445
> URL: https://issues.apache.org/jira/browse/OAK-3445
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.0.21, 1.3.6, 1.2.6
>Reporter: Julian Reschke
>Assignee: Julian Reschke
> Fix For: 1.3.7, 1.2.7, 1.0.22
>
>
> The WHERE clauses are currently generated in a generic fashion, thus
> including constraints on IDs that are known to evaluate to TRUE. This causes
> confusion when people look at SQL traces. It would help to skip constrains
> for {{ID > NodeDocument.MIN_ID_VALUE}} and {{ID < NodeDocument.MAX_ID_VALUE}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3445) RDBDocumentStore: when generating SQL for queries, leave out unneeded constraints
[
https://issues.apache.org/jira/browse/OAK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-3445:
Fix Version/s: 1.0.22
> RDBDocumentStore: when generating SQL for queries, leave out unneeded
> constraints
> -
>
> Key: OAK-3445
> URL: https://issues.apache.org/jira/browse/OAK-3445
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.0.21, 1.3.6, 1.2.6
>Reporter: Julian Reschke
>Assignee: Julian Reschke
> Fix For: 1.3.7, 1.2.7, 1.0.22
>
>
> The WHERE clauses are currently generated in a generic fashion, thus
> including constraints on IDs that are known to evaluate to TRUE. This causes
> confusion when people look at SQL traces. It would help to skip constrains
> for {{ID > NodeDocument.MIN_ID_VALUE}} and {{ID < NodeDocument.MAX_ID_VALUE}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3445) RDBDocumentStore: when generating SQL for queries, leave out unneeded constraints
[
https://issues.apache.org/jira/browse/OAK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-3445:
Fix Version/s: 1.2.7
> RDBDocumentStore: when generating SQL for queries, leave out unneeded
> constraints
> -
>
> Key: OAK-3445
> URL: https://issues.apache.org/jira/browse/OAK-3445
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.0.21, 1.3.6, 1.2.6
>Reporter: Julian Reschke
>Assignee: Julian Reschke
> Fix For: 1.3.7, 1.2.7
>
>
> The WHERE clauses are currently generated in a generic fashion, thus
> including constraints on IDs that are known to evaluate to TRUE. This causes
> confusion when people look at SQL traces. It would help to skip constrains
> for {{ID > NodeDocument.MIN_ID_VALUE}} and {{ID < NodeDocument.MAX_ID_VALUE}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3445) RDBDocumentStore: when generating SQL for queries, leave out unneeded constraints
[
https://issues.apache.org/jira/browse/OAK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-3445:
Fix Version/s: 1.3.7
> RDBDocumentStore: when generating SQL for queries, leave out unneeded
> constraints
> -
>
> Key: OAK-3445
> URL: https://issues.apache.org/jira/browse/OAK-3445
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.0.21, 1.3.6, 1.2.6
>Reporter: Julian Reschke
>Assignee: Julian Reschke
> Fix For: 1.3.7
>
>
> The WHERE clauses are currently generated in a generic fashion, thus
> including constraints on IDs that are known to evaluate to TRUE. This causes
> confusion when people look at SQL traces. It would help to skip constrains
> for {{ID > NodeDocument.MIN_ID_VALUE}} and {{ID < NodeDocument.MAX_ID_VALUE}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3445) RDBDocumentStore: when generating SQL for queries, leave out unneeded constraints
[
https://issues.apache.org/jira/browse/OAK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-3445:
Affects Version/s: 1.0.21
1.3.6
1.2.6
> RDBDocumentStore: when generating SQL for queries, leave out unneeded
> constraints
> -
>
> Key: OAK-3445
> URL: https://issues.apache.org/jira/browse/OAK-3445
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Affects Versions: 1.0.21, 1.3.6, 1.2.6
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>
> The WHERE clauses are currently generated in a generic fashion, thus
> including constraints on IDs that are known to evaluate to TRUE. This causes
> confusion when people look at SQL traces. It would help to skip constrains
> for {{ID > NodeDocument.MIN_ID_VALUE}} and {{ID < NodeDocument.MAX_ID_VALUE}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3445) RDBDocumentStore: when generating SQL for queries, leave out unneeded constraints
Julian Reschke created OAK-3445:
---
Summary: RDBDocumentStore: when generating SQL for queries, leave
out unneeded constraints
Key: OAK-3445
URL: https://issues.apache.org/jira/browse/OAK-3445
Project: Jackrabbit Oak
Issue Type: Technical task
Components: rdbmk
Reporter: Julian Reschke
The WHERE clauses are currently generated in a generic fashion, thus including
constraints on IDs that are known to evaluate to TRUE. This causes confusion
when people look at SQL traces. It would help to skip constrains for {{ID >
NodeDocument.MIN_ID_VALUE}} and {{ID < NodeDocument.MAX_ID_VALUE}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-3445) RDBDocumentStore: when generating SQL for queries, leave out unneeded constraints
[
https://issues.apache.org/jira/browse/OAK-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke reassigned OAK-3445:
---
Assignee: Julian Reschke
> RDBDocumentStore: when generating SQL for queries, leave out unneeded
> constraints
> -
>
> Key: OAK-3445
> URL: https://issues.apache.org/jira/browse/OAK-3445
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>
> The WHERE clauses are currently generated in a generic fashion, thus
> including constraints on IDs that are known to evaluate to TRUE. This causes
> confusion when people look at SQL traces. It would help to skip constrains
> for {{ID > NodeDocument.MIN_ID_VALUE}} and {{ID < NodeDocument.MAX_ID_VALUE}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2852) Query engine: if counter index is not available, cost of traversing is too low
[
https://issues.apache.org/jira/browse/OAK-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906059#comment-14906059
]
Thomas Mueller commented on OAK-2852:
-
http://svn.apache.org/r1705017 (rename internal methods)
> Query engine: if counter index is not available, cost of traversing is too low
> --
>
> Key: OAK-2852
> URL: https://issues.apache.org/jira/browse/OAK-2852
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: query
>Reporter: Thomas Mueller
>Assignee: Thomas Mueller
> Labels: performance
> Fix For: 1.3.7
>
>
> If the traversing index is not available (by removing or renaming the node
> /oak:index/counter), the cost of traversing is relatively low. This can cause
> traversals, even thought using a property index (or another index) would be
> better.
> By the way, disabling the counter index (setting the type to a 'disabled')
> alone does still use the estimation in the counter index. This may or may not
> be a good thing.
> Example costs:
> {noformat}
> /jcr:root/content//element(*, cq:Page)[@test='withCounter']
> cost for aggregate lucene is Infinity
> cost for lucene-property is Infinity
> cost for reference is Infinity
> cost for ordered is Infinity
> cost for nodeType is 138.0
> cost for property is Infinity
> cost for traverse is 27100.0
> /jcr:root/content//element(*, cq:Page)[@test='withoutCounter2']
> cost for aggregate lucene is Infinity
> cost for lucene-property is Infinity
> cost for reference is Infinity
> cost for ordered is Infinity
> cost for nodeType is 1504.0
> cost for property is Infinity
> cost for traverse is 2000.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3441) SecurityProviderImpl should not be an OSGi component
[ https://issues.apache.org/jira/browse/OAK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francesco Mari updated OAK-3441: Fix Version/s: 1.2.7 1.3.7 > SecurityProviderImpl should not be an OSGi component > > > Key: OAK-3441 > URL: https://issues.apache.org/jira/browse/OAK-3441 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: security >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.3.7, 1.2.7 > > > With the introduction of SecurityProviderRegistration in OAK-3201, > SecurityProviderImpl doesn't need to be registered as an OSGi component > anymore. SecurityProviderImpl should be turned into a plain class in a > backwards compatible way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3441) SecurityProviderImpl should not be an OSGi component
[ https://issues.apache.org/jira/browse/OAK-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906027#comment-14906027 ] Francesco Mari commented on OAK-3441: - Backported to 1.2 in r1705014. > SecurityProviderImpl should not be an OSGi component > > > Key: OAK-3441 > URL: https://issues.apache.org/jira/browse/OAK-3441 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: security >Reporter: Francesco Mari >Assignee: Francesco Mari > > With the introduction of SecurityProviderRegistration in OAK-3201, > SecurityProviderImpl doesn't need to be registered as an OSGi component > anymore. SecurityProviderImpl should be turned into a plain class in a > backwards compatible way. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3434) Revert backwards-incompatible changes to SecurityProviderImpl
[
https://issues.apache.org/jira/browse/OAK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906026#comment-14906026
]
Francesco Mari commented on OAK-3434:
-
Backported to 1.2 in r1705014.
> Revert backwards-incompatible changes to SecurityProviderImpl
> -
>
> Key: OAK-3434
> URL: https://issues.apache.org/jira/browse/OAK-3434
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: security
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7
>
> Attachments: OAK-3434-01.patch
>
>
> OAK-3201 introduced some backwards-incompatible changes to
> {{SecurityProviderImpl}}. It should be investigated if those changes can be
> reverted to maintain the backwards compatibility of the
> {{org.apache.jackrabbit.oak.security}} package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3431) SecurityProviderRegistration should not be part of an exported package
[ https://issues.apache.org/jira/browse/OAK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906025#comment-14906025 ] Francesco Mari commented on OAK-3431: - Backported to 1.2 in r1705014. > SecurityProviderRegistration should not be part of an exported package > -- > > Key: OAK-3431 > URL: https://issues.apache.org/jira/browse/OAK-3431 > Project: Jackrabbit Oak > Issue Type: Bug > Components: security >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.3.7, 1.2.7 > > > The SecurityProviderRegistration component should not be part of an exported > package. This component should be treated like an implementation detail of > the oak-core bundle, and no client code should directly depend on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3423) RandomAuthorizableNodeName should not be part of the default configuration of SecurityProviderImpl
[
https://issues.apache.org/jira/browse/OAK-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906024#comment-14906024
]
Francesco Mari commented on OAK-3423:
-
Backported to 1.2 in r1705014.
> RandomAuthorizableNodeName should not be part of the default configuration of
> SecurityProviderImpl
> --
>
> Key: OAK-3423
> URL: https://issues.apache.org/jira/browse/OAK-3423
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: security
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7
>
>
> OAK-3201 identified {{RandomAuthorizableNodeName}} as the default choice for
> {{SecurityProviderImpl}}. This choice is wrong and should be reverted. In
> particular, {{RandomAuthorizableNodeName}} should switch back to a required
> configuration policy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3201) Use static references in SecurityProviderImpl for composite services
[
https://issues.apache.org/jira/browse/OAK-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906023#comment-14906023
]
Francesco Mari commented on OAK-3201:
-
Backported to 1.2 in r1705014.
> Use static references in SecurityProviderImpl for composite services
>
>
> Key: OAK-3201
> URL: https://issues.apache.org/jira/browse/OAK-3201
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7
>
> Attachments: OAK-3201-01.patch, OAK-3201-02.patch, OAK-3201-03.patch,
> OAK-3201-04.patch, OAK-3201-05.patch, OAK-3201-06.patch, OAK-3201-07.patch,
> OAK-3201-08.patch, OAK-3201-09.patch, mbean-test.log
>
>
> {{SecurityProviderImpl}} has dynamic references to many other services, like
> {{RestrictionProvider}}, that represent the configuration of this component.
> Being these services dynamic, the OSGi runtime has no clear dependency
> relationship between the {{SecurityProviderImpl}} and the required services.
> Thus, it may happen that an instance of {{SecurityProviderImpl}} is published
> before the services it requires are started, creating a window where the
> {{SecurityProviderimpl}} is operating differently from the way it's
> configured.
> I suggest to turn the dynamic references in {{SecurityProviderImpl}} to
> static ones to improve the consistency of the implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3423) RandomAuthorizableNodeName should not be part of the default configuration of SecurityProviderImpl
[
https://issues.apache.org/jira/browse/OAK-3423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-3423:
Fix Version/s: 1.2.7
> RandomAuthorizableNodeName should not be part of the default configuration of
> SecurityProviderImpl
> --
>
> Key: OAK-3423
> URL: https://issues.apache.org/jira/browse/OAK-3423
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: security
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7
>
>
> OAK-3201 identified {{RandomAuthorizableNodeName}} as the default choice for
> {{SecurityProviderImpl}}. This choice is wrong and should be reverted. In
> particular, {{RandomAuthorizableNodeName}} should switch back to a required
> configuration policy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3434) Revert backwards-incompatible changes to SecurityProviderImpl
[
https://issues.apache.org/jira/browse/OAK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-3434:
Fix Version/s: 1.2.7
> Revert backwards-incompatible changes to SecurityProviderImpl
> -
>
> Key: OAK-3434
> URL: https://issues.apache.org/jira/browse/OAK-3434
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: security
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7
>
> Attachments: OAK-3434-01.patch
>
>
> OAK-3201 introduced some backwards-incompatible changes to
> {{SecurityProviderImpl}}. It should be investigated if those changes can be
> reverted to maintain the backwards compatibility of the
> {{org.apache.jackrabbit.oak.security}} package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3431) SecurityProviderRegistration should not be part of an exported package
[ https://issues.apache.org/jira/browse/OAK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francesco Mari updated OAK-3431: Fix Version/s: 1.2.7 > SecurityProviderRegistration should not be part of an exported package > -- > > Key: OAK-3431 > URL: https://issues.apache.org/jira/browse/OAK-3431 > Project: Jackrabbit Oak > Issue Type: Bug > Components: security >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.3.7, 1.2.7 > > > The SecurityProviderRegistration component should not be part of an exported > package. This component should be treated like an implementation detail of > the oak-core bundle, and no client code should directly depend on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3201) Use static references in SecurityProviderImpl for composite services
[
https://issues.apache.org/jira/browse/OAK-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-3201:
Fix Version/s: 1.2.7
> Use static references in SecurityProviderImpl for composite services
>
>
> Key: OAK-3201
> URL: https://issues.apache.org/jira/browse/OAK-3201
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.3.7, 1.2.7
>
> Attachments: OAK-3201-01.patch, OAK-3201-02.patch, OAK-3201-03.patch,
> OAK-3201-04.patch, OAK-3201-05.patch, OAK-3201-06.patch, OAK-3201-07.patch,
> OAK-3201-08.patch, OAK-3201-09.patch, mbean-test.log
>
>
> {{SecurityProviderImpl}} has dynamic references to many other services, like
> {{RestrictionProvider}}, that represent the configuration of this component.
> Being these services dynamic, the OSGi runtime has no clear dependency
> relationship between the {{SecurityProviderImpl}} and the required services.
> Thus, it may happen that an instance of {{SecurityProviderImpl}} is published
> before the services it requires are started, creating a window where the
> {{SecurityProviderimpl}} is operating differently from the way it's
> configured.
> I suggest to turn the dynamic references in {{SecurityProviderImpl}} to
> static ones to improve the consistency of the implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3436) Prevent missing checkpoint due to unstable topology from causing complete reindexing
[
https://issues.apache.org/jira/browse/OAK-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906005#comment-14906005
]
Chetan Mehrotra commented on OAK-3436:
--
bq. There is at least one element missing from this story: the lease stuff
Yup. Probably lease check logic is also affected by skew and does not see
latest state. Would try to come up with a testcase to reproduce the and support
the story!
bq. Second, if it's really about discovery taking some time to come online and
become consistent, is making the OSGi repository manager service dependent on
the OSGi discovery service an option
The issue was also seen on a normal running system. Topology can go bad due to
various reason. So we would need to be prepared all the time!
> Prevent missing checkpoint due to unstable topology from causing complete
> reindexing
>
>
> Key: OAK-3436
> URL: https://issues.apache.org/jira/browse/OAK-3436
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.3.7, 1.2.7, 1.0.22
>
>
> Async indexing logic relies on embedding application to ensure that async
> indexing job is run as a singleton in a cluster. For Sling based apps it
> depends on Sling Discovery support. At times it is being seen that if
> topology is not stable then different cluster nodes can consider them as
> leader and execute the async indexing job concurrently.
> This can cause problem as both cluster node might not see same repository
> state (due to write skew and eventual consistency) and might remove the
> checkpoint which other cluster node is still relying upon. For e.g. consider
> a 2 node cluster N1 and N2 where both are performing async indexing.
> # Base state - CP1 is the checkpoint for "async" job
> # N2 starts indexing and removes changes CP1 to CP2. For Mongo the
> checkpoints are saved in {{settings}} collection
> # N1 also decides to execute indexing but has yet not seen the latest
> repository state so still thinks that CP1 is the base checkpoint and tries to
> read it. However CP1 is already removed from {{settings}} and this makes N1
> think that checkpoint is missing and it decides to reindex everything!
> To avoid this topology must be stable but at Oak level we should still handle
> such a case and avoid doing a full reindexing. So we would need to have a
> {{MissingCheckpointStrategy}} similar to {{MissingIndexEditorStrategy}} as
> done in OAK-2203
> Possible approaches
> # A1 - Fail the indexing run if checkpoint is missing - Checkpoint being
> missing can have valid reason and invalid reason. Need to see what are valid
> scenarios where a checkpoint can go missing
> # A2 - When a checkpoint is created also store the creation time. When a
> checkpoint is found to be missing and its a *recent* checkpoint then fail the
> run. For e.g. we would fail the run till checkpoint found to be missing is
> less than an hour old (for just started take startup time into account)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3442) Intermittent IllegalMonitorStateException seen while releaseing IndexNode
[
https://issues.apache.org/jira/browse/OAK-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905997#comment-14905997
]
Chetan Mehrotra commented on OAK-3442:
--
bq. so there's a second error in the logs which is missing from the issue
description
The error came during startup upon first access to Lucene. So actual exception
got lost due to new exception. Below is the last exception seen before shutdown
{noformat}
21.09.2015 09:26:05.843 *ERROR* [pool-7-thread-2]
org.apache.sling.commons.scheduler.impl.QuartzScheduler Exception during job
execution of org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate@558f2a71
: Error occurred while obtaining InputStream for blobId
[e75b5be7d252e3b5ab87bf8a57c74ca06aff63a0#1047552]
java.lang.RuntimeException: Error occurred while obtaining InputStream for
blobId [e75b5be7d252e3b5ab87bf8a57c74ca06aff63a0#1047552]
at
org.apache.jackrabbit.oak.plugins.blob.BlobStoreBlob.getNewStream(BlobStoreBlob.java:49)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.getNewStream(SegmentBlob.java:84)
at
org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexFile.loadBlob(OakDirectory.java:216)
at
org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexFile.readBytes(OakDirectory.java:264)
at
org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexInput.readBytes(OakDirectory.java:350)
at
org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$OakIndexInput.readByte(OakDirectory.java:356)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
at
org.apache.lucene.store.CompoundFileDirectory.readEntries(CompoundFileDirectory.java:137)
at
org.apache.lucene.store.CompoundFileDirectory.(CompoundFileDirectory.java:104)
at
org.apache.lucene.index.SegmentReader.readFieldInfos(SegmentReader.java:202)
at
org.apache.lucene.index.IndexWriter.getFieldNumberMap(IndexWriter.java:818)
at org.apache.lucene.index.IndexWriter.(IndexWriter.java:770)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext.getWriter(LuceneIndexEditorContext.java:147)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.addOrUpdate(LuceneIndexEditor.java:266)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:176)
at
org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
at
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:153)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:418)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:583)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:418)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:583)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord$2.childNodeChanged(MapRecord.java:403)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord$3.childNodeChanged(MapRecord.java:444)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:487)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:436)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:394)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:583)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord$2.childNodeChanged(MapRecord.java:403)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:487)
at
org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:394)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:583)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
at
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate.updateIndex(AsyncIndexUpdate.java:367)
at
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate.run(AsyncIndexUpdate.java:312)
at
org.apache.sling.commons.scheduler.impl.QuartzJobExecutor.execute(QuartzJobExecutor.java:105)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
[jira] [Updated] (OAK-3443) Track the start time of mark in GC
[
https://issues.apache.org/jira/browse/OAK-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain updated OAK-3443:
---
Description:
Similarly to what is done for a shared datastore in OAK-3360, the start time of
the mark should be used as a reference for deletion of blobs in GC for 1.0.x.
The problem is that in 1.0.x the mark iterated over the data store which could
take a lot of time (many hours) and the buffer due to {{blobGcMaxAgeInSecs}}
may get stale due to which data loss could happen.
was:
Currently, for GC in a shared datastore, the last modified timestamp of the
earliest references file is used to calculate the max age of blobs to be
deleted. There is a possibility that the process itself could have taken quite
a long time which opens up a window that recent blobs could also be deleted.
The mark process is quite fast and with the default setting of
{{blobGcMaxAgeInSecs}} of 24 hours this should not be a problem but if the
{{blobGcMaxAgeInSec}} is specified to a lower value then it could create that
window described above.
> Track the start time of mark in GC
> --
>
> Key: OAK-3443
> URL: https://issues.apache.org/jira/browse/OAK-3443
> Project: Jackrabbit Oak
> Issue Type: Sub-task
> Components: blob
>Reporter: Amit Jain
>Assignee: Amit Jain
>Priority: Critical
> Labels: resilience
> Fix For: 1.0.22
>
>
> Similarly to what is done for a shared datastore in OAK-3360, the start time
> of the mark should be used as a reference for deletion of blobs in GC for
> 1.0.x. The problem is that in 1.0.x the mark iterated over the data store
> which could take a lot of time (many hours) and the buffer due to
> {{blobGcMaxAgeInSecs}} may get stale due to which data loss could happen.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905987#comment-14905987
]
Stefan Egli commented on OAK-3251:
--
bq. possible improvements in DocumentDiscoveryLiteServiceTest
I've created OAK-3444 to follow-up. Would take an overhaul of the whole test if
going virtual clock is tackled. I don't think it should be moved to ITs as it
then gets tested much less frequently probably.
> speeding up the build time
> --
>
> Key: OAK-3251
> URL: https://issues.apache.org/jira/browse/OAK-3251
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Attachments: build-1441353866-times.log, build-1441353866.log,
> build-1441637422.log, build-1443006309.log,
> oak-build-for-unittests-times.log, oak-build-times.log, times-1441637422.log,
> times-1443006309.log
>
>
> Running the build with a {mvn clean install} takes a considerable amount of
> time: 15 minutes on my local.
> The top 10 slowest unit test are the following
> {noformat}
> 110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 54.012 sec
> org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
> 36.593 sec
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
> 35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
> 24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> 17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
> 16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
> 14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
> 11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> {noformat}
> Is there anything we could do to speed-up these?
> sorted times obtained with
> https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3443) Track the start time of mark in GC
[
https://issues.apache.org/jira/browse/OAK-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain updated OAK-3443:
---
Fix Version/s: (was: 1.2.6)
(was: 1.3.6)
1.0.22
> Track the start time of mark in GC
> --
>
> Key: OAK-3443
> URL: https://issues.apache.org/jira/browse/OAK-3443
> Project: Jackrabbit Oak
> Issue Type: Sub-task
> Components: blob
>Reporter: Amit Jain
>Assignee: Amit Jain
>Priority: Minor
> Labels: resilience
> Fix For: 1.0.22
>
>
> Currently, for GC in a shared datastore, the last modified timestamp of the
> earliest references file is used to calculate the max age of blobs to be
> deleted. There is a possibility that the process itself could have taken
> quite a long time which opens up a window that recent blobs could also be
> deleted.
> The mark process is quite fast and with the default setting of
> {{blobGcMaxAgeInSecs}} of 24 hours this should not be a problem but if the
> {{blobGcMaxAgeInSec}} is specified to a lower value then it could create that
> window described above.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3444) Speed up DocumentDiscoveryLiteServiceTest
Stefan Egli created OAK-3444: Summary: Speed up DocumentDiscoveryLiteServiceTest Key: OAK-3444 URL: https://issues.apache.org/jira/browse/OAK-3444 Project: Jackrabbit Oak Issue Type: Technical task Components: core Affects Versions: 1.3.6 Reporter: Stefan Egli Currently the DocumentDiscoveryLiteServiceTest takes about 40-60sec - this would ideally be reduced to speed up the whole unit test part. * testLargeStartStopFiesta is the largest one with 17sec * perhaps using virtual clock instead of real clock could be an approach ** or make two variants: one for the ITs that uses real clock and one for the unit tests that uses virtual clocks.. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3443) Track the start time of mark in GC
[
https://issues.apache.org/jira/browse/OAK-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain updated OAK-3443:
---
Priority: Critical (was: Minor)
> Track the start time of mark in GC
> --
>
> Key: OAK-3443
> URL: https://issues.apache.org/jira/browse/OAK-3443
> Project: Jackrabbit Oak
> Issue Type: Sub-task
> Components: blob
>Reporter: Amit Jain
>Assignee: Amit Jain
>Priority: Critical
> Labels: resilience
> Fix For: 1.0.22
>
>
> Currently, for GC in a shared datastore, the last modified timestamp of the
> earliest references file is used to calculate the max age of blobs to be
> deleted. There is a possibility that the process itself could have taken
> quite a long time which opens up a window that recent blobs could also be
> deleted.
> The mark process is quite fast and with the default setting of
> {{blobGcMaxAgeInSecs}} of 24 hours this should not be a problem but if the
> {{blobGcMaxAgeInSec}} is specified to a lower value then it could create that
> window described above.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3443) Track the start time of mark in GC
Amit Jain created OAK-3443:
--
Summary: Track the start time of mark in GC
Key: OAK-3443
URL: https://issues.apache.org/jira/browse/OAK-3443
Project: Jackrabbit Oak
Issue Type: Sub-task
Components: blob
Reporter: Amit Jain
Assignee: Amit Jain
Priority: Minor
Fix For: 1.3.6, 1.2.6
Currently, for GC in a shared datastore, the last modified timestamp of the
earliest references file is used to calculate the max age of blobs to be
deleted. There is a possibility that the process itself could have taken quite
a long time which opens up a window that recent blobs could also be deleted.
The mark process is quite fast and with the default setting of
{{blobGcMaxAgeInSecs}} of 24 hours this should not be a problem but if the
{{blobGcMaxAgeInSec}} is specified to a lower value then it could create that
window described above.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3442) Intermittent IllegalMonitorStateException seen while releaseing IndexNode
[
https://issues.apache.org/jira/browse/OAK-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905956#comment-14905956
]
Alex Parvulescu commented on OAK-3442:
--
bq. So only way for such an exception to happen is that in a loop of say 2
paths indexNode got initialized for Loop 1 and then while acquiring in Loop 2
the indexNode still refers to old released value and that would cause the
exception.
so there's a second error in the logs which is missing from the issue
description then :)
bq. The fix should be simply to null the variable once released
Not to nitpick, but another solution could also be to move the variable
declaration inside the for block:
{code}
for (String path : indexPaths) {
IndexNode indexNode = null;
{code}
> Intermittent IllegalMonitorStateException seen while releaseing IndexNode
> -
>
> Key: OAK-3442
> URL: https://issues.apache.org/jira/browse/OAK-3442
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Fix For: 1.3.8
>
>
> At times following exception seen. On this system the index got corrupted
> because backing index files got deleted from the system and hence index is
> not accessible.
> {noformat}
> 21.09.2015 09:26:36.764 *ERROR* [FelixStartLevel]
> com.adobe.granite.repository.impl.SlingRepositoryManager start: Uncaught
> Throwable trying to access Repository, calling stopRepository()
> java.lang.IllegalMonitorStateException: attempt to unlock read lock, not
> locked by current thread
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.unmatchedUnlockException(ReentrantReadWriteLock.java:444)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryReleaseShared(ReentrantReadWriteLock.java:428)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.releaseShared(AbstractQueuedSynchronizer.java:1341)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.unlock(ReentrantReadWriteLock.java:881)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.release(IndexNode.java:121)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.getPlans(LucenePropertyIndex.java:212)
> at
> org.apache.jackrabbit.oak.query.QueryImpl.getBestSelectorExecutionPlan(QueryImpl.java:847)
> at
> org.apache.jackrabbit.oak.query.QueryImpl.getBestSelectorExecutionPlan(QueryImpl.java:793)
> at
> org.apache.jackrabbit.oak.query.ast.SelectorImpl.prepare(SelectorImpl.java:283)
> at
> org.apache.jackrabbit.oak.query.QueryImpl.prepare(QueryImpl.java:568)
> at
> org.apache.jackrabbit.oak.query.QueryEngineImpl.executeQuery(QueryEngineImpl.java:183)
> at
> org.apache.jackrabbit.oak.security.user.UserProvider.getAuthorizableByPrincipal(UserProvider.java:234)
> at
> org.apache.jackrabbit.oak.security.user.UserManagerImpl.getAuthorizable(UserManagerImpl.java:116)
> at
> org.apache.jackrabbit.oak.security.principal.PrincipalProviderImpl.getAuthorizable(PrincipalProviderImpl.java:140)
> at
> org.apache.jackrabbit.oak.security.principal.PrincipalProviderImpl.getPrincipal(PrincipalProviderImpl.java:69)
> at
> org.apache.jackrabbit.oak.spi.security.principal.CompositePrincipalProvider.getPrincipal(CompositePrincipalProvider.java:50)
> at
> org.apache.jackrabbit.oak.spi.security.principal.PrincipalManagerImpl.getPrincipal(PrincipalManagerImpl.java:47)
> at
> com.adobe.granite.repository.impl.SlingRepositoryManager.setupPermissions(SlingRepositoryManager.java:997)
> at
> com.adobe.granite.repository.impl.SlingRepositoryManager.createRepository(SlingRepositoryManager.java:420)
> at
> com.adobe.granite.repository.impl.SlingRepositoryManager.acquireRepository(SlingRepositoryManager.java:290)
> at
> org.apache.sling.jcr.base.AbstractSlingRepositoryManager.start(AbstractSlingRepositoryManager.java:304)
> at
> com.adobe.granite.repository.impl.SlingRepositoryManager.activate(SlingRepositoryManager.java:267)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at
> org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:222)
> at
> org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
> at
> org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:615)
>
[jira] [Commented] (OAK-3436) Prevent missing checkpoint due to unstable topology from causing complete reindexing
[
https://issues.apache.org/jira/browse/OAK-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14905939#comment-14905939
]
Alex Parvulescu commented on OAK-3436:
--
There is at least one element missing from this story: the lease stuff, which I
think was introduced to provide a low-level fallback against different nodes
indexing at the same time, linking to code, I don't have an oak issue, sorry
[0]. Is this mechanism no longer effective?
Second, if it's really about discovery taking some time to come online and
become consistent, is making the OSGi repository manager service dependent on
the OSGi discovery service an option? (outside of oak-core of course, as oak
has no direct knowledge of discovery bits).
[0]
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/index/AsyncIndexUpdate.java#L192
> Prevent missing checkpoint due to unstable topology from causing complete
> reindexing
>
>
> Key: OAK-3436
> URL: https://issues.apache.org/jira/browse/OAK-3436
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: query
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.3.7, 1.2.7, 1.0.22
>
>
> Async indexing logic relies on embedding application to ensure that async
> indexing job is run as a singleton in a cluster. For Sling based apps it
> depends on Sling Discovery support. At times it is being seen that if
> topology is not stable then different cluster nodes can consider them as
> leader and execute the async indexing job concurrently.
> This can cause problem as both cluster node might not see same repository
> state (due to write skew and eventual consistency) and might remove the
> checkpoint which other cluster node is still relying upon. For e.g. consider
> a 2 node cluster N1 and N2 where both are performing async indexing.
> # Base state - CP1 is the checkpoint for "async" job
> # N2 starts indexing and removes changes CP1 to CP2. For Mongo the
> checkpoints are saved in {{settings}} collection
> # N1 also decides to execute indexing but has yet not seen the latest
> repository state so still thinks that CP1 is the base checkpoint and tries to
> read it. However CP1 is already removed from {{settings}} and this makes N1
> think that checkpoint is missing and it decides to reindex everything!
> To avoid this topology must be stable but at Oak level we should still handle
> such a case and avoid doing a full reindexing. So we would need to have a
> {{MissingCheckpointStrategy}} similar to {{MissingIndexEditorStrategy}} as
> done in OAK-2203
> Possible approaches
> # A1 - Fail the indexing run if checkpoint is missing - Checkpoint being
> missing can have valid reason and invalid reason. Need to see what are valid
> scenarios where a checkpoint can go missing
> # A2 - When a checkpoint is created also store the creation time. When a
> checkpoint is found to be missing and its a *recent* checkpoint then fail the
> run. For e.g. we would fail the run till checkpoint found to be missing is
> less than an hour old (for just started take startup time into account)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3442) Intermittent IllegalMonitorStateException seen while releaseing IndexNode
Chetan Mehrotra created OAK-3442:
Summary: Intermittent IllegalMonitorStateException seen while
releaseing IndexNode
Key: OAK-3442
URL: https://issues.apache.org/jira/browse/OAK-3442
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Priority: Minor
Fix For: 1.3.8
At times following exception seen. On this system the index got corrupted
because backing index files got deleted from the system and hence index is not
accessible.
{noformat}
21.09.2015 09:26:36.764 *ERROR* [FelixStartLevel]
com.adobe.granite.repository.impl.SlingRepositoryManager start: Uncaught
Throwable trying to access Repository, calling stopRepository()
java.lang.IllegalMonitorStateException: attempt to unlock read lock, not locked
by current thread
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.unmatchedUnlockException(ReentrantReadWriteLock.java:444)
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryReleaseShared(ReentrantReadWriteLock.java:428)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.releaseShared(AbstractQueuedSynchronizer.java:1341)
at
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.unlock(ReentrantReadWriteLock.java:881)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.release(IndexNode.java:121)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.getPlans(LucenePropertyIndex.java:212)
at
org.apache.jackrabbit.oak.query.QueryImpl.getBestSelectorExecutionPlan(QueryImpl.java:847)
at
org.apache.jackrabbit.oak.query.QueryImpl.getBestSelectorExecutionPlan(QueryImpl.java:793)
at
org.apache.jackrabbit.oak.query.ast.SelectorImpl.prepare(SelectorImpl.java:283)
at org.apache.jackrabbit.oak.query.QueryImpl.prepare(QueryImpl.java:568)
at
org.apache.jackrabbit.oak.query.QueryEngineImpl.executeQuery(QueryEngineImpl.java:183)
at
org.apache.jackrabbit.oak.security.user.UserProvider.getAuthorizableByPrincipal(UserProvider.java:234)
at
org.apache.jackrabbit.oak.security.user.UserManagerImpl.getAuthorizable(UserManagerImpl.java:116)
at
org.apache.jackrabbit.oak.security.principal.PrincipalProviderImpl.getAuthorizable(PrincipalProviderImpl.java:140)
at
org.apache.jackrabbit.oak.security.principal.PrincipalProviderImpl.getPrincipal(PrincipalProviderImpl.java:69)
at
org.apache.jackrabbit.oak.spi.security.principal.CompositePrincipalProvider.getPrincipal(CompositePrincipalProvider.java:50)
at
org.apache.jackrabbit.oak.spi.security.principal.PrincipalManagerImpl.getPrincipal(PrincipalManagerImpl.java:47)
at
com.adobe.granite.repository.impl.SlingRepositoryManager.setupPermissions(SlingRepositoryManager.java:997)
at
com.adobe.granite.repository.impl.SlingRepositoryManager.createRepository(SlingRepositoryManager.java:420)
at
com.adobe.granite.repository.impl.SlingRepositoryManager.acquireRepository(SlingRepositoryManager.java:290)
at
org.apache.sling.jcr.base.AbstractSlingRepositoryManager.start(AbstractSlingRepositoryManager.java:304)
at
com.adobe.granite.repository.impl.SlingRepositoryManager.activate(SlingRepositoryManager.java:267)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at
org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:222)
at
org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:37)
at
org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:615)
at
org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:499)
at
org.apache.felix.scr.impl.helper.ActivateMethod.invoke(ActivateMethod.java:295)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:302)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:113)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:832)
at
org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:799)
at
org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:724)
at
org.apache.felix.scr.impl.manager.DependencyManager$SingleStaticCustomizer.addedService(DependencyManager.java:927)
at
org.apache.felix.scr.impl.manager.Depende
