BUILD FAILURE: Jackrabbit Oak - Build # 246 - Failure
The Apache Jenkins build system has built Jackrabbit Oak (build #246) Status: Failure Check console output at https://builds.apache.org/job/Jackrabbit%20Oak/246/ to view the results. Changes: [chetanm] OAK-5558 - Consistency checker for Lucene indexes Implemented support for testing based on CheckIndex utility from Lucene [chetanm] OAK-5558 - Consistency checker for Lucene indexes Implemented basic check which checks for all referred blobs being present and valid Test results: All tests passed
Sort order in MongoMissingLastRevSeeker
Hi, I recently noticed the sort order in MongoMissingLastRevSeeker on _modified. The query sorts descending. I think this may be a problem given the isolation guarantees provides by MongoDB. See also discussion in OAK-4535. In particular, I'm thinking of the following behaviour mentioned in the MongoDB documentation: "Reads may miss matching documents that are updated during the course of the read operation." This happens when a query scans an index and a matching document is concurrently updated and moves from the front of the cursor to the part of the index that was already scanned. In this case MongoDB will not return the document even if it matched the query before and after the update. Given the query in MongoMissingLastRevSeeker, it is quite likely MongoDB will scan the _modified index from (currently) newest to specified startTime. While doing this, a document may get updated, its index entry jumps over the cursor and becomes the newest entry in the index. A can't really think of a reason why the sort order must be descending in MongoMissingLastRevSeeker. The base class doesn't specify a specific order. Does anyone remember why this sort order is in place? Regards Marcel
MongoMK failover behaviour.
Hi, What is the expected behaviour when a Oak MongoMK experiences a MongoDB primary failure. I am looking at an instance that appears to try and retry reads repeatedly from the MongoDB primary and after 60s or more reports the Oak Discovery lease has been lost, resulting in many minutes of retries there eventually shutting down the repository. I don't currently have the MongoDB logs to share. Just wondering what to expect at this stage ? I am starting from the assumption that Oak works perfectly in this regard. Best Regards Ian
Re: MongoMK failover behaviour.
Hi, On 04/05/17 12:02, Ian Boston wrote: What is the expected behaviour when a Oak MongoMK experiences a MongoDB primary failure. I am looking at an instance that appears to try and retry reads repeatedly from the MongoDB primary and after 60s or more reports the Oak Discovery lease has been lost, resulting in many minutes of retries there eventually shutting down the repository. I don't currently have the MongoDB logs to share. Just wondering what to expect at this stage ? Oak will stop the oak-core bundle if a MongoDB primary is unavailable for more than 110 seconds. The 110 seconds are based on the 120 seconds lease timeout and a lease update interval of 10 seconds. When this happens, all reads and writes to the repository will fail. Though, in an OSGi environment services depending on oak-core should stop as well. You will need to restart the system or affected bundles once the primary is available again. See also discussion in OAK-3397 and OAK-3250. Regards Marcel
Re: Sort order in MongoMissingLastRevSeeker
On Thu, May 4, 2017 at 3:15 PM, Marcel Reutegger wrote: > A can't really think of a reason why the sort order must be descending in > MongoMissingLastRevSeeker. I also do not remember. I think it should be fine to drop the sort criteria. Chetan Mehrotra
Re: MongoMK failover behaviour.
Hi, On 4 May 2017 at 11:26, Marcel Reutegger wrote: > Hi, > > On 04/05/17 12:02, Ian Boston wrote: > >> What is the expected behaviour when a Oak MongoMK experiences a MongoDB >> primary failure. >> >> I am looking at an instance that appears to try and retry reads repeatedly >> from the MongoDB primary and after 60s or more reports the Oak Discovery >> lease has been lost, resulting in many minutes of retries there eventually >> shutting down the repository. >> >> I don't currently have the MongoDB logs to share. Just wondering what to >> expect at this stage ? >> > > Oak will stop the oak-core bundle if a MongoDB primary is unavailable for > more than 110 seconds. The 110 seconds are based on the 120 seconds lease > timeout and a lease update interval of 10 seconds. > Yes, that happens after 120s. Before 120 seconds, should the MongoDB Java driver route read queries to a secondary and use the new primary without any action by Oak (eg closing a connection and opening a new one ) ? Best Regards Ian > > When this happens, all reads and writes to the repository will fail. > Though, in an OSGi environment services depending on oak-core should stop > as well. You will need to restart the system or affected bundles once the > primary is available again. See also discussion in OAK-3397 and OAK-3250. > > Regards > Marcel >
Re: MongoMK failover behaviour.
Hi, On 04/05/17 14:57, Ian Boston wrote: Before 120 seconds, should the MongoDB Java driver route read queries to a secondary and use the new primary without any action by Oak (eg closing a connection and opening a new one ) ? Yes, the MongoDB Java driver automatically routes queries based on their required read preference. The failover is automatic and the driver should direct queries to the new primary once available. Connection pooling is done by the driver. Oak does not manage those. Regards Marcel
Re: MongoMK failover behaviour.
Hi, On 4 May 2017 at 15:19, Marcel Reutegger wrote: > Hi, > > On 04/05/17 14:57, Ian Boston wrote: > >> Before 120 seconds, should the MongoDB Java driver route read queries to a >> secondary and use the new primary without any action by Oak (eg closing a >> connection and opening a new one ) ? >> > > Yes, the MongoDB Java driver automatically routes queries based on their > required read preference. The failover is automatic and the driver should > direct queries to the new primary once available. Connection pooling is > done by the driver. Oak does not manage those. > Thanks. Looks like there might be a problem with the MongoDB deployment in the case I am looking at. Either due to performance or misconfiguration. Dropping a primary results in read queries failing and after 120s the Oak repositories shutdown as they are not able to write. All that points to the MongoDB driver config, or the MongoDB instances and not Oak. Best Regards Ian > > Regards > Marcel >
Re: MongoMK failover behaviour.
Hi, On 04/05/17 16:35, Ian Boston wrote: Looks like there might be a problem with the MongoDB deployment in the case I am looking at. Either due to performance or misconfiguration. Dropping a primary results in read queries failing and after 120s the Oak repositories shutdown as they are not able to write. All that points to the MongoDB driver config, or the MongoDB instances and not Oak. Hmm, depending on the Oak version, this may also be caused by OAK-5528. The current fix versions are 1.4.15 and 1.6.0. Regards Marcel
Re: MongoMK failover behaviour.
Hi, On Thu, May 4, 2017 at 10:46 AM Marcel Reutegger wrote: > Hi, > > On 04/05/17 16:35, Ian Boston wrote: > > Looks like there might be a problem with the MongoDB deployment in the > case > > I am looking at. Either due to performance or misconfiguration. Dropping > a > > primary results in read queries failing and after 120s the Oak > repositories > > shutdown as they are not able to write. All that points to the MongoDB > > driver config, or the MongoDB instances and not Oak. > > Hmm, depending on the Oak version, this may also be caused by OAK-5528. > The current fix versions are 1.4.15 and 1.6.0. > Would this show up in thread dumps? Based on the description, it seems like it should. Regards, Justin > > Regards > Marcel >
BUILD FAILURE: Jackrabbit Oak - Build # 253 - Failure
The Apache Jenkins build system has built Jackrabbit Oak (build #253) Status: Failure Check console output at https://builds.apache.org/job/Jackrabbit%20Oak/253/ to view the results. Changes: [angela] OAK-5882 : Improve coverage for oak.security code in oak-core Test results: 1 tests failed. FAILED: org.apache.jackrabbit.oak.plugins.document.VersionGCTest.gcMonitorStatusUpdates Error Message: expected:<[INITIALIZING, COLLECTING, UPDATING, SPLITS_CLEANUP, IDLE]> but was:<[INITIALIZING, COLLECTING, CHECKING, COLLECTING, DELETING, SORTING, DELETING, UPDATING, SPLITS_CLEANUP, IDLE]> Stack Trace: java.lang.AssertionError: expected:<[INITIALIZING, COLLECTING, UPDATING, SPLITS_CLEANUP, IDLE]> but was:<[INITIALIZING, COLLECTING, CHECKING, COLLECTING, DELETING, SORTING, DELETING, UPDATING, SPLITS_CLEANUP, IDLE]> at org.apache.jackrabbit.oak.plugins.document.VersionGCTest.gcMonitorStatusUpdates(VersionGCTest.java:207)
[ANNOUNCE] Apache Jackrabbit 2.15.2 released
The Apache Jackrabbit community is pleased to announce the release of Apache Jackrabbit 2.15.2. The release is available for download at: https://jackrabbit.apache.org/jcr/downloads.html#v2.15 See the full release notes below for details about this release: Release Notes -- Apache Jackrabbit -- Version 2.15.2 Introduction This is Apache Jackrabbit(TM) 2.15.2, a fully compliant implementation of the Content Repository for Java(TM) Technology API, version 2.0 (JCR 2.0) as specified in the Java Specification Request 283 (JSR 283). Apache Jackrabbit 2.15.2 is an unstable release cut directly from Jackrabbit trunk, with a focus on new features and other improvements. For production use we recommend the latest stable 2.14.x release. Changes in Jackrabbit 2.15.2 Bug [JCR-4118] - RepositoryChecker creates invalid node names [JCR-4121] - ConcurrentModificationException in InternalVersionHistoryImpl.fixLegacy() [JCR-4133] - fix javadoc problems that are errors with JDK8 Task [JCR-4112] - Require Java 8 [JCR-4119] - Upgrade httpcomponents/httpmime to 4.5.3 [JCR-4122] - align parent pom references with Oak [JCR-4127] - update to latest apache parent pom (18) [JCR-4128] - update maven plugins and require Maven 3.2.1 [JCR-4129] - get rid of unused org.json dependency In addition to the above-mentioned changes, this release contains all the changes included up to the Apache Jackrabbit 2.14.x release. For more detailed information about all the changes in this and other Jackrabbit releases, please see the Jackrabbit issue tracker at https://issues.apache.org/jira/browse/JCR Release Contents This release consists of a single source archive packaged as a zip file. The archive can be unpacked with the jar tool from your JDK installation. See the README.txt file for instructions on how to build this release. The source archive is accompanied by SHA1 and MD5 checksums and a PGP signature that you can use to verify the authenticity of your download. The public key used for the PGP signature can be found at https://svn.apache.org/repos/asf/jackrabbit/dist/KEYS. About Apache Jackrabbit --- Apache Jackrabbit is a fully conforming implementation of the Content Repository for Java Technology API (JCR). A content repository is a hierarchical content store with support for structured and unstructured content, full text search, versioning, transactions, observation, and more. For more information, visit http://jackrabbit.apache.org/ About The Apache Software Foundation Established in 1999, The Apache Software Foundation provides organizational, legal, and financial support for more than 140 freely-available, collaboratively-developed Open Source projects. The pragmatic Apache License enables individual and commercial users to easily deploy Apache software; the Foundation's intellectual property framework limits the legal exposure of its 3,800+ contributors. For more information, visit http://www.apache.org/ Trademarks -- Apache Jackrabbit, Jackrabbit, Apache, the Apache feather logo, and the Apache Jackrabbit project logo are trademarks of The Apache Software Foundation.
