[jira] [Resolved] (OAK-3670) RDBDocumentStore on SQLServer: off-by-one bug may cause truncated JSON to be written
[ https://issues.apache.org/jira/browse/OAK-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-3670. - Resolution: Fixed trunk: http://svn.apache.org/r1715888 and http://svn.apache.org/r1715898 1.2: http://svn.apache.org/r1715901 1.0: http://svn.apache.org/r1715902 > RDBDocumentStore on SQLServer: off-by-one bug may cause truncated JSON to be > written > > > Key: OAK-3670 > URL: https://issues.apache.org/jira/browse/OAK-3670 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.3.11, 1.2.8, 1.0.24 >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.2.9, 1.0.25, 1.3.12 > > > In the SQLServer-specific append logic, an off-by-one bug might cause JSON > data to be truncated thus becoming invalid. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3368) Speed up ExternalPrivateStoreIT and ExternalSharedStoreIT
[ https://issues.apache.org/jira/browse/OAK-3368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-3368: -- Fix Version/s: (was: 1.3.12) 1.4 > Speed up ExternalPrivateStoreIT and ExternalSharedStoreIT > - > > Key: OAK-3368 > URL: https://issues.apache.org/jira/browse/OAK-3368 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: tarmk-standby >Reporter: Marcel Reutegger >Assignee: Manfred Baedke > Fix For: 1.4 > > > Both tests run for more than 5 minutes. Most of the time the tests are > somehow stuck in shutting down the server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2066) DocumentStore API: batch create, but no batch update
[ https://issues.apache.org/jira/browse/OAK-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021764#comment-15021764 ] Tomek Rękawek commented on OAK-2066: Patches should be applied in following order: 1. OAK-3586 ConflictException and CommitQueue should support a list of revisions 2. OAK-3662 Create bulk createOrUpdate method and use it in Commit 3a. OAK-3559 for Mongo 3b. OAK-3637 for RDB > DocumentStore API: batch create, but no batch update > > > Key: OAK-2066 > URL: https://issues.apache.org/jira/browse/OAK-2066 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: mongomk >Reporter: Julian Reschke > Labels: performance > > The DocumentStore API currently has a call for creating many nodes at once. > However, this will sometimes fail for large save operations in JCR, because > in the DS persistence, JCR-deleted nodes are still present (with a deleted > flag). This causes two subsequent sequences of > 1) create test container > 2) create many child nodes > 3) remove test container > to behave very differently, depending on whether the test container is > created for the first time or not. > (see CreateManyChildNodesTest) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3662) Create bulk createOrUpdate method and use it in Commit
[
https://issues.apache.org/jira/browse/OAK-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-3662:
---
Description: The {{DocumentStore#createOrUpdate(Collection, UpdateOp)}}
method is invoked in a loop in the {{Commit#applyToDocumentStore()}}, once for
each changed node. Investigate if it's possible to implement a batch version of
the createOrUpdate method. It should return all documents before they are
modified, so the Commit class can discover conflicts (if they are any).
> Create bulk createOrUpdate method and use it in Commit
> --
>
> Key: OAK-3662
> URL: https://issues.apache.org/jira/browse/OAK-3662
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: documentmk
>Reporter: Tomek Rękawek
> Fix For: 1.4
>
>
> The {{DocumentStore#createOrUpdate(Collection, UpdateOp)}} method is invoked
> in a loop in the {{Commit#applyToDocumentStore()}}, once for each changed
> node. Investigate if it's possible to implement a batch version of the
> createOrUpdate method. It should return all documents before they are
> modified, so the Commit class can discover conflicts (if they are any).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3559) Bulk document updates in MongoDocumentStore
[ https://issues.apache.org/jira/browse/OAK-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomek Rękawek updated OAK-3559: --- Component/s: (was: documentmk) (was: core) > Bulk document updates in MongoDocumentStore > --- > > Key: OAK-3559 > URL: https://issues.apache.org/jira/browse/OAK-3559 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: mongomk >Reporter: Tomek Rękawek > Fix For: 1.4 > > Attachments: OAK-3559.patch > > > Using the MongoDB [Bulk > API|https://docs.mongodb.org/manual/reference/method/Bulk/#Bulk] implement > the [batch version of createOrUpdate method|OAK-3662]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3355) Test failure: SpellcheckTest.testSpellcheckMultipleWords
[
https://issues.apache.org/jira/browse/OAK-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger updated OAK-3355:
--
Description:
{{org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords}}
fails on Jenkins.
Failure seen at builds: 389, 392, 395, 396, 562
https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/396/jdk=jdk-1.6u45,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
{noformat}
testSpellcheckMultipleWords(org.apache.jackrabbit.oak.jcr.query.SpellcheckTest)
Time elapsed: 0.907 sec <<< FAILURE!
junit.framework.ComparisonFailure: expected:<[voting[ in] ontario]> but
was:<[voting[, voted,] ontario]>
at junit.framework.Assert.assertEquals(Assert.java:85)
at junit.framework.Assert.assertEquals(Assert.java:91)
at
org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords(SpellcheckTest.java:86)
{noformat}
was:
{{org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords}}
fails on Jenkins.
Failure seen at builds: 389, 392, 395, 396
https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/396/jdk=jdk-1.6u45,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
{noformat}
testSpellcheckMultipleWords(org.apache.jackrabbit.oak.jcr.query.SpellcheckTest)
Time elapsed: 0.907 sec <<< FAILURE!
junit.framework.ComparisonFailure: expected:<[voting[ in] ontario]> but
was:<[voting[, voted,] ontario]>
at junit.framework.Assert.assertEquals(Assert.java:85)
at junit.framework.Assert.assertEquals(Assert.java:91)
at
org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords(SpellcheckTest.java:86)
{noformat}
> Test failure: SpellcheckTest.testSpellcheckMultipleWords
>
>
> Key: OAK-3355
> URL: https://issues.apache.org/jira/browse/OAK-3355
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
> Environment:
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
> Labels: ci, jenkins, test-failure
> Fix For: 1.4
>
>
> {{org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords}}
> fails on Jenkins.
> Failure seen at builds: 389, 392, 395, 396, 562
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/396/jdk=jdk-1.6u45,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> {noformat}
> testSpellcheckMultipleWords(org.apache.jackrabbit.oak.jcr.query.SpellcheckTest)
> Time elapsed: 0.907 sec <<< FAILURE!
> junit.framework.ComparisonFailure: expected:<[voting[ in] ontario]> but
> was:<[voting[, voted,] ontario]>
> at junit.framework.Assert.assertEquals(Assert.java:85)
> at junit.framework.Assert.assertEquals(Assert.java:91)
> at
> org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords(SpellcheckTest.java:86)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3355) Test failure: SpellcheckTest.testSpellcheckMultipleWords
[
https://issues.apache.org/jira/browse/OAK-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021967#comment-15021967
]
Marcel Reutegger commented on OAK-3355:
---
Added test to known issues list again in trunk: http://svn.apache.org/r1715767
See recent test failure for trunk:
https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/jdk=latest1.7,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=unittesting/562/console
{noformat}
Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 2.9 sec <<<
FAILURE!
testSpellcheckMultipleWords(org.apache.jackrabbit.oak.jcr.query.SpellcheckTest)
Time elapsed: 0.969 sec <<< FAILURE!
junit.framework.ComparisonFailure: expected:<[voting in ontario]> but
was:<[[voting, voted, ontario]]>
{noformat}
> Test failure: SpellcheckTest.testSpellcheckMultipleWords
>
>
> Key: OAK-3355
> URL: https://issues.apache.org/jira/browse/OAK-3355
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
> Environment:
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
> Labels: ci, jenkins, test-failure
> Fix For: 1.4
>
>
> {{org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords}}
> fails on Jenkins.
> Failure seen at builds: 389, 392, 395, 396, 562
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/396/jdk=jdk-1.6u45,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> {noformat}
> testSpellcheckMultipleWords(org.apache.jackrabbit.oak.jcr.query.SpellcheckTest)
> Time elapsed: 0.907 sec <<< FAILURE!
> junit.framework.ComparisonFailure: expected:<[voting[ in] ontario]> but
> was:<[voting[, voted,] ontario]>
> at junit.framework.Assert.assertEquals(Assert.java:85)
> at junit.framework.Assert.assertEquals(Assert.java:91)
> at
> org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords(SpellcheckTest.java:86)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3665) Oak Run TarMK revision diff
Alex Parvulescu created OAK-3665: Summary: Oak Run TarMK revision diff Key: OAK-3665 URL: https://issues.apache.org/jira/browse/OAK-3665 Project: Jackrabbit Oak Issue Type: Improvement Components: run, segmentmk Reporter: Alex Parvulescu Assignee: Alex Parvulescu Fix For: 1.3.11 I'd like to add support for generating diffs between 2 given revisions, either direct or incrementally (one by one between the 2 given limits). Oak Explorer already has partial support for this, this issue is for the oak-run support to generate a log with the diff output. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-3665) Oak Run TarMK revision diff
[ https://issues.apache.org/jira/browse/OAK-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu resolved OAK-3665. -- Resolution: Fixed > Oak Run TarMK revision diff > --- > > Key: OAK-3665 > URL: https://issues.apache.org/jira/browse/OAK-3665 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run, segmentmk >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.3.11 > > > I'd like to add support for generating diffs between 2 given revisions, > either direct or incrementally (one by one between the 2 given limits). > Oak Explorer already has partial support for this, this issue is for the > oak-run support to generate a log with the diff output. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3649) Extract node document cache from Mongo and RDB document stores
[ https://issues.apache.org/jira/browse/OAK-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021965#comment-15021965 ] Chetan Mehrotra commented on OAK-3649: -- bq. I saw that you've recently worked in this area, removing the HierrachialCacheInvalidator. Could you take a look on this patch and apply it if you'll find it useful? [~tomek.rekawek] Cache logic is pretty involved so any refactoring there needs to be done carefully. [~mreutegg] Would you be looking into the patch and apply it ... or let me know if you want me to do that > Extract node document cache from Mongo and RDB document stores > -- > > Key: OAK-3649 > URL: https://issues.apache.org/jira/browse/OAK-3649 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk, mongomk, rdbmk >Reporter: Tomek Rękawek >Priority: Minor > Labels: candidate_oak_1_0, candidate_oak_1_2 > Fix For: 1.3.12 > > > MongoDocumentStore and RDBDocumentStore contains copy & pasted methods > responsible for handling node document cache. Extract these into a new > NodeDocumentCache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-3355) Test failure: SpellcheckTest.testSpellcheckMultipleWords
[
https://issues.apache.org/jira/browse/OAK-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcel Reutegger reassigned OAK-3355:
-
Assignee: Tommaso Teofili
> Test failure: SpellcheckTest.testSpellcheckMultipleWords
>
>
> Key: OAK-3355
> URL: https://issues.apache.org/jira/browse/OAK-3355
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
> Environment:
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>Assignee: Tommaso Teofili
> Labels: ci, jenkins, test-failure
> Fix For: 1.4
>
>
> {{org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords}}
> fails on Jenkins.
> Failure seen at builds: 389, 392, 395, 396, 562
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/396/jdk=jdk-1.6u45,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> {noformat}
> testSpellcheckMultipleWords(org.apache.jackrabbit.oak.jcr.query.SpellcheckTest)
> Time elapsed: 0.907 sec <<< FAILURE!
> junit.framework.ComparisonFailure: expected:<[voting[ in] ontario]> but
> was:<[voting[, voted,] ontario]>
> at junit.framework.Assert.assertEquals(Assert.java:85)
> at junit.framework.Assert.assertEquals(Assert.java:91)
> at
> org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords(SpellcheckTest.java:86)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3661) RDBDocumentStore: improve logging for invalid data in persistence
[ https://issues.apache.org/jira/browse/OAK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3661: Fix Version/s: 1.3.12 > RDBDocumentStore: improve logging for invalid data in persistence > - > > Key: OAK-3661 > URL: https://issues.apache.org/jira/browse/OAK-3661 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.2.7, 1.3.10, 1.0.23 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Fix For: 1.3.12 > > > When data can not be read due to malformed JSON in the database, we minimally > need to log the document ID as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3662) Create bulk createOrUpdate method and use it in Commit
Tomek Rękawek created OAK-3662: -- Summary: Create bulk createOrUpdate method and use it in Commit Key: OAK-3662 URL: https://issues.apache.org/jira/browse/OAK-3662 Project: Jackrabbit Oak Issue Type: Technical task Components: documentmk Reporter: Tomek Rękawek Fix For: 1.4 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3637) Bulk document updates in RDBDocumentStore
[ https://issues.apache.org/jira/browse/OAK-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomek Rękawek updated OAK-3637: --- Description: Implement the [batch createOrUpdate|OAK-3662] in the RDBDocumentStore. (was: OAK-3559 contains implementation of the bulk createOrUpdate() method for MongoDocumentStore. Similar method should be implemented in the RDBDocumentStore.) > Bulk document updates in RDBDocumentStore > - > > Key: OAK-3637 > URL: https://issues.apache.org/jira/browse/OAK-3637 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Reporter: Tomek Rękawek > Fix For: 1.4 > > > Implement the [batch createOrUpdate|OAK-3662] in the RDBDocumentStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-3661) RDBDocumentStore: improve logging for invalid data in persistence
[ https://issues.apache.org/jira/browse/OAK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-3661. - Resolution: Fixed trunk: http://svn.apache.org/r1715771 1.2: http://svn.apache.org/r1715772 1.0: http://svn.apache.org/r1715776 > RDBDocumentStore: improve logging for invalid data in persistence > - > > Key: OAK-3661 > URL: https://issues.apache.org/jira/browse/OAK-3661 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.2.7, 1.3.10, 1.0.23 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Fix For: 1.2.9, 1.0.25, 1.3.12 > > > When data can not be read due to malformed JSON in the database, we minimally > need to log the document ID as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-3666) Add information about repeated upgrade to documentation
[ https://issues.apache.org/jira/browse/OAK-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli resolved OAK-3666. -- Resolution: Fixed applied patch - public site https://jackrabbit.apache.org/oak/docs/migration.html is now up to date. > Add information about repeated upgrade to documentation > --- > > Key: OAK-3666 > URL: https://issues.apache.org/jira/browse/OAK-3666 > Project: Jackrabbit Oak > Issue Type: Documentation > Components: doc, upgrade >Reporter: Tomek Rękawek >Assignee: Stefan Egli > Fix For: 1.3.11 > > Attachments: OAK-3666.patch > > > Please add info about the OAK-2619 to the [migration documentation > page|https://jackrabbit.apache.org/oak/docs/migration.html]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-3355) Test failure: SpellcheckTest.testSpellcheckMultipleWords
[
https://issues.apache.org/jira/browse/OAK-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Saurabh reassigned OAK-3355:
--
Assignee: Vikas Saurabh (was: Tommaso Teofili)
> Test failure: SpellcheckTest.testSpellcheckMultipleWords
>
>
> Key: OAK-3355
> URL: https://issues.apache.org/jira/browse/OAK-3355
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
> Environment:
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>Assignee: Vikas Saurabh
> Labels: ci, jenkins, test-failure
> Fix For: 1.4
>
>
> {{org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords}}
> fails on Jenkins.
> Failure seen at builds: 389, 392, 395, 396, 562
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/396/jdk=jdk-1.6u45,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> {noformat}
> testSpellcheckMultipleWords(org.apache.jackrabbit.oak.jcr.query.SpellcheckTest)
> Time elapsed: 0.907 sec <<< FAILURE!
> junit.framework.ComparisonFailure: expected:<[voting[ in] ontario]> but
> was:<[voting[, voted,] ontario]>
> at junit.framework.Assert.assertEquals(Assert.java:85)
> at junit.framework.Assert.assertEquals(Assert.java:91)
> at
> org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords(SpellcheckTest.java:86)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-3666) Add information about repeated upgrade to documentation
[ https://issues.apache.org/jira/browse/OAK-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli reassigned OAK-3666: Assignee: Stefan Egli > Add information about repeated upgrade to documentation > --- > > Key: OAK-3666 > URL: https://issues.apache.org/jira/browse/OAK-3666 > Project: Jackrabbit Oak > Issue Type: Documentation > Components: doc, upgrade >Reporter: Tomek Rękawek >Assignee: Stefan Egli > Fix For: 1.3.11 > > Attachments: OAK-3666.patch > > > Please add info about the OAK-2619 to the [migration documentation > page|https://jackrabbit.apache.org/oak/docs/migration.html]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3661) RDBDocumentStore: improve logging for invalid data in persistence
[ https://issues.apache.org/jira/browse/OAK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3661: Fix Version/s: 1.2.9 > RDBDocumentStore: improve logging for invalid data in persistence > - > > Key: OAK-3661 > URL: https://issues.apache.org/jira/browse/OAK-3661 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.2.7, 1.3.10, 1.0.23 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Fix For: 1.2.9, 1.3.12 > > > When data can not be read due to malformed JSON in the database, we minimally > need to log the document ID as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3654) Integrate with Metrics for various stats collection
[ https://issues.apache.org/jira/browse/OAK-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021991#comment-15021991 ] Chetan Mehrotra commented on OAK-3654: -- bq. nanoTimer and performance. I have measured the overhead of the metrics library to be about 10% timing calls of about 1E-7s. [~ianeboston] Coming on this again. Oak has a Clock.Fast implementation as it was observed that fetching nanoTime can lead to slowdown on some setup. See [discussion in OAK-1418|https://issues.apache.org/jira/browse/OAK-1418?focusedCommentId=13923093=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13923093] for more details. I would try to run a benchmark and see if the number are bad or good. However it very much depend on hardware so might not get detected. As metrics has a Clock abstraction we can provide our own implementation there based [Clock.Fast|https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/stats/Clock.java#L280]. Thoughts? > Integrate with Metrics for various stats collection > > > Key: OAK-3654 > URL: https://issues.apache.org/jira/browse/OAK-3654 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: core >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.4 > > Attachments: OAK-3654-v1.patch, query-stats.png > > > As suggested by [~ianeboston] in OAK-3478 current approach of collecting > TimeSeries data is not easily consumable by other monitoring systems. Also > just extracting the last moment data and exposing it in simple form would not > be useful. > Instead of that we should look into using Metrics library [1] for collecting > metrics. To avoid having dependency on Metrics API all over in Oak we can > come up with minimal interfaces which can be used in Oak and then provide an > implementation backed by Metric. > This task is meant to explore that aspect and come up with proposed changes > to see if its feasible to make this change > * metrics-core ~100KB in size with no dependency > * ASL Licensee > [1] http://metrics.dropwizard.io -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3667) Refactor executor closing logic as a utility class
Chetan Mehrotra created OAK-3667:
Summary: Refactor executor closing logic as a utility class
Key: OAK-3667
URL: https://issues.apache.org/jira/browse/OAK-3667
Project: Jackrabbit Oak
Issue Type: Task
Components: commons, core
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Priority: Minor
Fix For: 1.3.12
Logic in {{ExecutorCloser}} in Oak class is duplicated in
RepositoryImpl#closeExecutor. This is also required now in
{{StatisticsProviderFactory}}.
For that it should be refactored out as a utility class in
oak-commons/org.apache.jackrabbit.oak.commons.concurrent
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3654) Integrate with Metrics for various stats collection
[ https://issues.apache.org/jira/browse/OAK-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022031#comment-15022031 ] Ian Boston commented on OAK-3654: - Clock.Fast has a maximum resolution of 1ms and when getTime is called it makes a call to System.nanoTime() and a call to System.getCurrentTimeMillis(), hence when ACCURATE.getTime() is called it will generally be slower than just calling System.nanoTime(). The result from Clock.Fast.getTime() is of little use in the Metrics use case. Reading the background I think the requirements from Oak for Clock.Fast are different from the requirements from a Metrics use case. A metrics use case must have nano second time resolution and should not require more than 1 call to System.nanoTime() to achieve that. IIUC Oak needed ms time resolution. I don't believe there is any requirement to deal with long term drift in time (ie 1% or less) over a period of > 10s. If the OS/JDK really is very slow for System.nanoTime() calls then it's probably a JDK bug. I see the testes were done on JDK 7u45 which is very old compared to the current release. If there is a issue with non linearity in time or steps in time then its likely the OS ntpd is misconfigured. If you read https://blogs.oracle.com/dholmes/entry/inside_the_hotspot_vm_clocks it explains why System.getCurrentTimeMillis() is faster than System.nanoTime() (measured at 13ns vs 18ns). It also explains why the 2 drift, and strongly recommends using System.nanoTime() to measure intervals, as that call generally binds to a crystal oscillator with sub 1% drift. The blog post also warns about mixing the 2 sources of time. There are some notes specific to the Windows impl where System.getCurrentTimeMillis() has a 10ms resolution for certain versions for the JDK. So in answer to your question: Metrics has considerable production experience. I would avoid making fundamental changes unilaterally. If there is a real problem with the Metrics code and how they measure time, I would open an issue on their bug tracker to validate the issue is real. > Integrate with Metrics for various stats collection > > > Key: OAK-3654 > URL: https://issues.apache.org/jira/browse/OAK-3654 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: core >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.4 > > Attachments: OAK-3654-v1.patch, query-stats.png > > > As suggested by [~ianeboston] in OAK-3478 current approach of collecting > TimeSeries data is not easily consumable by other monitoring systems. Also > just extracting the last moment data and exposing it in simple form would not > be useful. > Instead of that we should look into using Metrics library [1] for collecting > metrics. To avoid having dependency on Metrics API all over in Oak we can > come up with minimal interfaces which can be used in Oak and then provide an > implementation backed by Metric. > This task is meant to explore that aspect and come up with proposed changes > to see if its feasible to make this change > * metrics-core ~100KB in size with no dependency > * ASL Licensee > [1] http://metrics.dropwizard.io -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OAK-3355) Test failure: SpellcheckTest.testSpellcheckMultipleWords
[
https://issues.apache.org/jira/browse/OAK-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vikas Saurabh reassigned OAK-3355:
--
Assignee: Tommaso Teofili (was: Vikas Saurabh)
Assigning back to Tommaso. Initially, I thought that the failure were due to my
changes in OAK-3509 - but it seems that's not the case.
> Test failure: SpellcheckTest.testSpellcheckMultipleWords
>
>
> Key: OAK-3355
> URL: https://issues.apache.org/jira/browse/OAK-3355
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: solr
> Environment:
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/
>Reporter: Michael Dürig
>Assignee: Tommaso Teofili
> Labels: ci, jenkins, test-failure
> Fix For: 1.4
>
>
> {{org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords}}
> fails on Jenkins.
> Failure seen at builds: 389, 392, 395, 396, 562
> https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/396/jdk=jdk-1.6u45,label=Ubuntu,nsfixtures=DOCUMENT_RDB,profile=unittesting/console
> {noformat}
> testSpellcheckMultipleWords(org.apache.jackrabbit.oak.jcr.query.SpellcheckTest)
> Time elapsed: 0.907 sec <<< FAILURE!
> junit.framework.ComparisonFailure: expected:<[voting[ in] ontario]> but
> was:<[voting[, voted,] ontario]>
> at junit.framework.Assert.assertEquals(Assert.java:85)
> at junit.framework.Assert.assertEquals(Assert.java:91)
> at
> org.apache.jackrabbit.oak.jcr.query.SpellcheckTest.testSpellcheckMultipleWords(SpellcheckTest.java:86)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3559) Bulk document updates in MongoDocumentStore
[
https://issues.apache.org/jira/browse/OAK-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-3559:
---
Description: Using the MongoDB [Bulk
API|https://docs.mongodb.org/manual/reference/method/Bulk/#Bulk] implement the
[batch version of createOrUpdate method|OAK-3662]. (was: The
{{DocumentStore#createOrUpdate(Collection, UpdateOp)}} method is invoked in a
loop in the {{Commit#applyToDocumentStore()}}, once for each changed node.
Investigate if it's possible to implement a batch version of the createOrUpdate
method, using the MongoDB [Bulk
API|https://docs.mongodb.org/manual/reference/method/Bulk/#Bulk]. It should
return all documents before they are modified, so the Commit class can discover
conflicts (if they are any).)
> Bulk document updates in MongoDocumentStore
> ---
>
> Key: OAK-3559
> URL: https://issues.apache.org/jira/browse/OAK-3559
> Project: Jackrabbit Oak
> Issue Type: Sub-task
> Components: core, documentmk, mongomk
>Reporter: Tomek Rękawek
> Fix For: 1.4
>
> Attachments: OAK-3559.patch
>
>
> Using the MongoDB [Bulk
> API|https://docs.mongodb.org/manual/reference/method/Bulk/#Bulk] implement
> the [batch version of createOrUpdate method|OAK-3662].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3662) Create bulk createOrUpdate method and use it in Commit
[
https://issues.apache.org/jira/browse/OAK-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-3662:
---
Attachment: OAK-3662.patch
> Create bulk createOrUpdate method and use it in Commit
> --
>
> Key: OAK-3662
> URL: https://issues.apache.org/jira/browse/OAK-3662
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: documentmk
>Reporter: Tomek Rękawek
> Fix For: 1.4
>
> Attachments: OAK-3662.patch
>
>
> The {{DocumentStore#createOrUpdate(Collection, UpdateOp)}} method is invoked
> in a loop in the {{Commit#applyToDocumentStore()}}, once for each changed
> node. Investigate if it's possible to implement a batch version of the
> createOrUpdate method. It should return all documents before they are
> modified, so the Commit class can discover conflicts (if they are any).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3662) Create bulk createOrUpdate method and use it in Commit
[
https://issues.apache.org/jira/browse/OAK-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021831#comment-15021831
]
Tomek Rękawek commented on OAK-3662:
Patch attached.
Branch on github:
https://github.com/trekawek/jackrabbit-oak/tree/OAK-3662
> Create bulk createOrUpdate method and use it in Commit
> --
>
> Key: OAK-3662
> URL: https://issues.apache.org/jira/browse/OAK-3662
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: documentmk
>Reporter: Tomek Rękawek
> Fix For: 1.4
>
> Attachments: OAK-3662.patch
>
>
> The {{DocumentStore#createOrUpdate(Collection, UpdateOp)}} method is invoked
> in a loop in the {{Commit#applyToDocumentStore()}}, once for each changed
> node. Investigate if it's possible to implement a batch version of the
> createOrUpdate method. It should return all documents before they are
> modified, so the Commit class can discover conflicts (if they are any).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3559) Bulk document updates in MongoDocumentStore
[ https://issues.apache.org/jira/browse/OAK-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomek Rękawek updated OAK-3559: --- Attachment: (was: OAK-3559.patch) > Bulk document updates in MongoDocumentStore > --- > > Key: OAK-3559 > URL: https://issues.apache.org/jira/browse/OAK-3559 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: mongomk >Reporter: Tomek Rękawek > Fix For: 1.4 > > > Using the MongoDB [Bulk > API|https://docs.mongodb.org/manual/reference/method/Bulk/#Bulk] implement > the [batch version of createOrUpdate method|OAK-3662]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3663) LastRevRecoveryRandomizedIT fails with seed 10848868
Marcel Reutegger created OAK-3663: - Summary: LastRevRecoveryRandomizedIT fails with seed 10848868 Key: OAK-3663 URL: https://issues.apache.org/jira/browse/OAK-3663 Project: Jackrabbit Oak Issue Type: Bug Components: core, documentmk Affects Versions: 1.3.11 Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor See recent build on travis: https://travis-ci.org/apache/jackrabbit-oak/builds/92654149 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3663) LastRevRecoveryRandomizedIT fails with seed 10848868
[ https://issues.apache.org/jira/browse/OAK-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-3663: -- Fix Version/s: 1.4 > LastRevRecoveryRandomizedIT fails with seed 10848868 > > > Key: OAK-3663 > URL: https://issues.apache.org/jira/browse/OAK-3663 > Project: Jackrabbit Oak > Issue Type: Bug > Components: core, documentmk >Affects Versions: 1.3.11 >Reporter: Marcel Reutegger >Assignee: Marcel Reutegger >Priority: Minor > Fix For: 1.4 > > > See recent build on travis: > https://travis-ci.org/apache/jackrabbit-oak/builds/92654149 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3559) Bulk document updates in MongoDocumentStore
[ https://issues.apache.org/jira/browse/OAK-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomek Rękawek updated OAK-3559: --- Attachment: OAK-3559.patch Refactored patch to include only the changes in the MongoDocumentStore. The rest of the code has been extracted to OAK-3662. > Bulk document updates in MongoDocumentStore > --- > > Key: OAK-3559 > URL: https://issues.apache.org/jira/browse/OAK-3559 > Project: Jackrabbit Oak > Issue Type: Sub-task > Components: mongomk >Reporter: Tomek Rękawek > Fix For: 1.4 > > Attachments: OAK-3559.patch > > > Using the MongoDB [Bulk > API|https://docs.mongodb.org/manual/reference/method/Bulk/#Bulk] implement > the [batch version of createOrUpdate method|OAK-3662]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3637) Bulk document updates in RDBDocumentStore
[ https://issues.apache.org/jira/browse/OAK-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomek Rękawek updated OAK-3637: --- Flags: Patch > Bulk document updates in RDBDocumentStore > - > > Key: OAK-3637 > URL: https://issues.apache.org/jira/browse/OAK-3637 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Reporter: Tomek Rękawek > Fix For: 1.4 > > Attachments: OAK-3637.patch > > > Implement the [batch createOrUpdate|OAK-3662] in the RDBDocumentStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3649) Extract node document cache from Mongo and RDB document stores
[ https://issues.apache.org/jira/browse/OAK-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15021914#comment-15021914 ] Marcel Reutegger commented on OAK-3649: --- TreeLocks were initially introduced with OAK-1897 because concurrent update() and query() calls resulted in stale documents in the cache. > Extract node document cache from Mongo and RDB document stores > -- > > Key: OAK-3649 > URL: https://issues.apache.org/jira/browse/OAK-3649 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk, mongomk, rdbmk >Reporter: Tomek Rękawek >Priority: Minor > Labels: candidate_oak_1_0, candidate_oak_1_2 > Fix For: 1.3.12 > > > MongoDocumentStore and RDBDocumentStore contains copy & pasted methods > responsible for handling node document cache. Extract these into a new > NodeDocumentCache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3661) RDBDocumentStore: improve logging for invalid data in persistence
Julian Reschke created OAK-3661: --- Summary: RDBDocumentStore: improve logging for invalid data in persistence Key: OAK-3661 URL: https://issues.apache.org/jira/browse/OAK-3661 Project: Jackrabbit Oak Issue Type: Technical task Components: documentmk Affects Versions: 1.0.23, 1.3.10, 1.2.7 Reporter: Julian Reschke Assignee: Julian Reschke Priority: Minor When data can not be read due to malformed JSON in the database, we minimally need to log the document ID as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3637) Bulk document updates in RDBDocumentStore
[ https://issues.apache.org/jira/browse/OAK-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomek Rękawek updated OAK-3637: --- Attachment: OAK-3637.patch Patch attached. Branch: https://github.com/trekawek/jackrabbit-oak/tree/OAK-3637 > Bulk document updates in RDBDocumentStore > - > > Key: OAK-3637 > URL: https://issues.apache.org/jira/browse/OAK-3637 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Reporter: Tomek Rękawek > Fix For: 1.4 > > Attachments: OAK-3637.patch > > > Implement the [batch createOrUpdate|OAK-3662] in the RDBDocumentStore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3662) Create bulk createOrUpdate method and use it in Commit
[
https://issues.apache.org/jira/browse/OAK-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-3662:
---
Flags: Patch
> Create bulk createOrUpdate method and use it in Commit
> --
>
> Key: OAK-3662
> URL: https://issues.apache.org/jira/browse/OAK-3662
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: documentmk
>Reporter: Tomek Rękawek
> Fix For: 1.4
>
> Attachments: OAK-3662.patch
>
>
> The {{DocumentStore#createOrUpdate(Collection, UpdateOp)}} method is invoked
> in a loop in the {{Commit#applyToDocumentStore()}}, once for each changed
> node. Investigate if it's possible to implement a batch version of the
> createOrUpdate method. It should return all documents before they are
> modified, so the Commit class can discover conflicts (if they are any).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3661) RDBDocumentStore: improve logging for invalid data in persistence
[ https://issues.apache.org/jira/browse/OAK-3661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3661: Component/s: (was: documentmk) rdbmk > RDBDocumentStore: improve logging for invalid data in persistence > - > > Key: OAK-3661 > URL: https://issues.apache.org/jira/browse/OAK-3661 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.2.7, 1.3.10, 1.0.23 >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > > When data can not be read due to malformed JSON in the database, we minimally > need to log the document ID as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3654) Integrate with Metrics for various stats collection
[ https://issues.apache.org/jira/browse/OAK-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15023775#comment-15023775 ] Chetan Mehrotra commented on OAK-3654: -- bq. hence when ACCURATE.getTime() is called it will generally be slower than just calling System.nanoTime(). Ack bq. it explains why System.getCurrentTimeMillis() is faster than System.nanoTime() (measured at 13ns vs 18ns). Yes thats the understanding. Key point here is that Metrics is being called in very critical path of read and write and if every such call involves a call to System.nanoTime then that raise a bit of concern. Now looking back at the change done so far I think there is some extra duplicate work now. Currently we maintain a pair of metrics * SESSION_READ_COUNTER and SESSION_READ_DURATION * SESSION_WRITE_COUNTER and SESSION_WRITE_DURATION * QUERY_COUNT and QUERY_DURATION In current approach for each pair we manage a pair of Meter and Timer. Hence any call now involve 2 calls to System.nanoTime(). Would have to refactor that to avoid the extra call bq. Metrics has considerable production experience. I would avoid making fundamental changes unilaterally. If there is a real problem with the Metrics code and how they measure time, I would open an issue on their bug tracker to validate the issue is real. Sure. Would try to get some numbers to benchmark > Integrate with Metrics for various stats collection > > > Key: OAK-3654 > URL: https://issues.apache.org/jira/browse/OAK-3654 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: core >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.4 > > Attachments: OAK-3654-v1.patch, query-stats.png > > > As suggested by [~ianeboston] in OAK-3478 current approach of collecting > TimeSeries data is not easily consumable by other monitoring systems. Also > just extracting the last moment data and exposing it in simple form would not > be useful. > Instead of that we should look into using Metrics library [1] for collecting > metrics. To avoid having dependency on Metrics API all over in Oak we can > come up with minimal interfaces which can be used in Oak and then provide an > implementation backed by Metric. > This task is meant to explore that aspect and come up with proposed changes > to see if its feasible to make this change > * metrics-core ~100KB in size with no dependency > * ASL Licensee > [1] http://metrics.dropwizard.io -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (OAK-3654) Integrate with Metrics for various stats collection
[ https://issues.apache.org/jira/browse/OAK-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15023775#comment-15023775 ] Chetan Mehrotra edited comment on OAK-3654 at 11/24/15 6:40 AM: bq. hence when ACCURATE.getTime() is called it will generally be slower than just calling System.nanoTime(). Ack bq. it explains why System.getCurrentTimeMillis() is faster than System.nanoTime() (measured at 13ns vs 18ns). Yes thats the understanding. Key point here is that Metrics is being called in very critical path of read and write and if every such call involves a call to System.nanoTime then that raise a bit of concern. Now looking back at the change done so far I think there is some extra duplicate work now. Currently we maintain a pair of metrics * SESSION_READ_COUNTER and SESSION_READ_DURATION * SESSION_WRITE_COUNTER and SESSION_WRITE_DURATION * QUERY_COUNT and QUERY_DURATION In current approach for each pair we manage a pair of Meter and Timer. Hence any call now involve 4 calls to System.nanoTime() # one to determine duration, # 1 in Meter like SESSION_READ_COUNTER. Meter does a call for every mark in tickIfNecessary - This can be saved by turning it into a NoopMeter with r1716032 # 1 in Meter managed within the Timer like SESSION_READ_DURATION # 1 in Histrogram (via ExponentiallyDecayingReservoir) being managed within the Timer Would have to refactor that to avoid the extra call. But with that also it would get down to 3 bq. Metrics has considerable production experience. I would avoid making fundamental changes unilaterally. If there is a real problem with the Metrics code and how they measure time, I would open an issue on their bug tracker to validate the issue is real. Sure. Would try to get some numbers to benchmark. btw looking at the issue tracker for Metric came across this interesting thread on [Mechanical Sympathy|https://groups.google.com/forum/#!msg/mechanical-sympathy/I4JfZQ1GYi8] which is very much related to latency measurement we are doing here. For very fast operation say 1000/sec it would be better to go for sampling to measure the timing. Also have a look at http://latencyutils.github.io/LatencyUtils/ So may be we should not do measurement in the very critical parts like read and write average. Still digging deeper was (Author: chetanm): bq. hence when ACCURATE.getTime() is called it will generally be slower than just calling System.nanoTime(). Ack bq. it explains why System.getCurrentTimeMillis() is faster than System.nanoTime() (measured at 13ns vs 18ns). Yes thats the understanding. Key point here is that Metrics is being called in very critical path of read and write and if every such call involves a call to System.nanoTime then that raise a bit of concern. Now looking back at the change done so far I think there is some extra duplicate work now. Currently we maintain a pair of metrics * SESSION_READ_COUNTER and SESSION_READ_DURATION * SESSION_WRITE_COUNTER and SESSION_WRITE_DURATION * QUERY_COUNT and QUERY_DURATION In current approach for each pair we manage a pair of Meter and Timer. Hence any call now involve 3 calls to System.nanoTime() # one to determine duration, # 1 in Meter like SESSION_READ_COUNTER. Meter does a call for every mark in tickIfNecessary - This can be saved by turning it into a NoopMeter with r1716032 # 1 in Meter managed within the Timer like SESSION_READ_DURATION Would have to refactor that to avoid the extra call. But with that also it would get down to 2 bq. Metrics has considerable production experience. I would avoid making fundamental changes unilaterally. If there is a real problem with the Metrics code and how they measure time, I would open an issue on their bug tracker to validate the issue is real. Sure. Would try to get some numbers to benchmark > Integrate with Metrics for various stats collection > > > Key: OAK-3654 > URL: https://issues.apache.org/jira/browse/OAK-3654 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: core >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.4 > > Attachments: OAK-3654-v1.patch, query-stats.png > > > As suggested by [~ianeboston] in OAK-3478 current approach of collecting > TimeSeries data is not easily consumable by other monitoring systems. Also > just extracting the last moment data and exposing it in simple form would not > be useful. > Instead of that we should look into using Metrics library [1] for collecting > metrics. To avoid having dependency on Metrics API all over in Oak we can > come up with minimal interfaces which can be used in Oak and then provide an > implementation backed by Metric. > This task is meant to explore that aspect and come up
[jira] [Commented] (OAK-3654) Integrate with Metrics for various stats collection
[ https://issues.apache.org/jira/browse/OAK-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15023912#comment-15023912 ] Ian Boston commented on OAK-3654: - My tests showed that timing measurements of operations that took less than 1E-7s contained a 10% overhead which concurs with the observation that System.nanoTime() takes order 1E-8s to complete. If you need to measure metrics at that level, then using a Timer wrapping an individual call is the wrong approach as you will need a timer with an overhead of more like 1E-9 or 1E-10s and an accuracy of 1E-8 or 1E-9s. Realistically, continuous and routine timing at that level is not particularly useful in production and best left to attaching a profiler with sampling support. > Integrate with Metrics for various stats collection > > > Key: OAK-3654 > URL: https://issues.apache.org/jira/browse/OAK-3654 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: core >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra > Fix For: 1.4 > > Attachments: OAK-3654-v1.patch, query-stats.png > > > As suggested by [~ianeboston] in OAK-3478 current approach of collecting > TimeSeries data is not easily consumable by other monitoring systems. Also > just extracting the last moment data and exposing it in simple form would not > be useful. > Instead of that we should look into using Metrics library [1] for collecting > metrics. To avoid having dependency on Metrics API all over in Oak we can > come up with minimal interfaces which can be used in Oak and then provide an > implementation backed by Metric. > This task is meant to explore that aspect and come up with proposed changes > to see if its feasible to make this change > * metrics-core ~100KB in size with no dependency > * ASL Licensee > [1] http://metrics.dropwizard.io -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-3621) Required property type not respected
[
https://issues.apache.org/jira/browse/OAK-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
angela resolved OAK-3621.
-
Resolution: Fixed
Assignee: angela
Fix Version/s: 1.3.11
> Required property type not respected
>
>
> Key: OAK-3621
> URL: https://issues.apache.org/jira/browse/OAK-3621
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, jcr
>Reporter: Antonio Sanso
>Assignee: angela
> Fix For: 1.3.11
>
> Attachments: OAK-3621_2.patch, mixin_definition.png, property_type.png
>
>
> It seems like Section 7.1.5 Adding and Writing Properties of jsr 170
> {code}
> Sets the specified (single
> value) property of this node
> to the specified value. If the property does not yet
> exist, it is created. The property type of the property
> will be that specified by the node type of this node.
> If the property type of the
> supplied Value object is
> different from that required, then a best-effort
> conversion is attempted. If the conversion fails, a
> ValueFormatException is thrown. If another error
> occurs, a RepositoryException is thrown.
> If the node type of this node
> does not indicate a
> specific property type, then the property type of the
> supplied Value object is used and if the property
> already exists (has previously been set) it assumes
> both the new value and new property type.
> {code}
> is not respected.In the provided screenshots you can see a mixin definition
> having a long property. But the property type in the creation node is Boolean
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-3632) Incorrect Value Conversion upon Node.setProperty and Node.setValue
[ https://issues.apache.org/jira/browse/OAK-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angela resolved OAK-3632. - Resolution: Fixed Committed revision 1715893. [~asanso], feel free to reopen, if you find something not covered by the fix. thanks. > Incorrect Value Conversion upon Node.setProperty and Node.setValue > -- > > Key: OAK-3632 > URL: https://issues.apache.org/jira/browse/OAK-3632 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: jcr >Reporter: angela > Attachments: OAK-3632.patch > > > see container issue for test-cases and description -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3670) RDBDocumentStore on SQLServer: off-by-one bug may cause truncated JSON to be written
[ https://issues.apache.org/jira/browse/OAK-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3670: Fix Version/s: 1.2.9 > RDBDocumentStore on SQLServer: off-by-one bug may cause truncated JSON to be > written > > > Key: OAK-3670 > URL: https://issues.apache.org/jira/browse/OAK-3670 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.3.11, 1.0.23, 1.2.8 >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.2.9, 1.3.12 > > > In the SQLServer-specific append logic, an off-by-one bug might cause JSON > data to be truncated thus becoming invalid. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3670) RDBDocumentStore on SQLServer: off-by-one bug may cause truncated JSON to be written
[ https://issues.apache.org/jira/browse/OAK-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3670: Affects Version/s: (was: 1.3.10) (was: 1.2.7) 1.2.8 1.3.11 > RDBDocumentStore on SQLServer: off-by-one bug may cause truncated JSON to be > written > > > Key: OAK-3670 > URL: https://issues.apache.org/jira/browse/OAK-3670 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.3.11, 1.0.23, 1.2.8 >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.2.9, 1.3.12 > > > In the SQLServer-specific append logic, an off-by-one bug might cause JSON > data to be truncated thus becoming invalid. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3670) RDBDocumentStore on SQLServer: off-by-one bug may cause truncated JSON to be written
[ https://issues.apache.org/jira/browse/OAK-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3670: Fix Version/s: 1.3.12 > RDBDocumentStore on SQLServer: off-by-one bug may cause truncated JSON to be > written > > > Key: OAK-3670 > URL: https://issues.apache.org/jira/browse/OAK-3670 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: rdbmk >Affects Versions: 1.2.7, 1.3.10, 1.0.23 >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.3.12 > > > In the SQLServer-specific append logic, an off-by-one bug might cause JSON > data to be truncated thus becoming invalid. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3668) Potential test failure: CompactionAndCleanupIT#testMixedSegments
Michael Dürig created OAK-3668:
--
Summary: Potential test failure:
CompactionAndCleanupIT#testMixedSegments
Key: OAK-3668
URL: https://issues.apache.org/jira/browse/OAK-3668
Project: Jackrabbit Oak
Issue Type: Bug
Components: segmentmk
Reporter: Michael Dürig
Assignee: Michael Dürig
{{CompactionAndCleanupIT#testMixedSegments}} might fail under some
circumstances. It can be certainly be made to fail by increasing concurrency. I
suspect this to be caused by OAK-3348.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3637) Bulk document updates in RDBDocumentStore
[
https://issues.apache.org/jira/browse/OAK-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022202#comment-15022202
]
Tomek Rękawek commented on OAK-3637:
I've run the {{CreateManyChildNodesTest}} benchmark, using the MySQL. The numer
of children created in one iteration is set to 100 (I wanted to have at least a
few iterations during a 5-minute test). Results are as follows:
{noformat}
### latency: 0ms, bulk size: 100 ###
C min 10% 50% 90% max N
bulk (OAK-3637) 1 87 92 98 118 4501577
sequential (SNAPSHOT) 1 368 375 397 461 862 470
### latency: 20ms, bulk size: 100 ###
C min 10% 50% 90% max N
bulk (OAK-3637) 177267754787279477973 22
sequential (SNAPSHOT) 1 42639 42639 42686 42769 42769 5
{noformat}
> Bulk document updates in RDBDocumentStore
> -
>
> Key: OAK-3637
> URL: https://issues.apache.org/jira/browse/OAK-3637
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Tomek Rękawek
> Fix For: 1.4
>
> Attachments: OAK-3637.patch
>
>
> Implement the [batch createOrUpdate|OAK-3662] in the RDBDocumentStore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3668) Potential test failure: CompactionAndCleanupIT#testMixedSegments
[
https://issues.apache.org/jira/browse/OAK-3668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15022212#comment-15022212
]
Michael Dürig commented on OAK-3668:
Setting OAK-3348 as blocker. We need to revisit this issue once that one is
fixed.
> Potential test failure: CompactionAndCleanupIT#testMixedSegments
>
>
> Key: OAK-3668
> URL: https://issues.apache.org/jira/browse/OAK-3668
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: compaction, gc
>
> {{CompactionAndCleanupIT#testMixedSegments}} might fail under some
> circumstances. It can be certainly be made to fail by increasing concurrency.
> I suspect this to be caused by OAK-3348.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
