[jira] [Created] (OAK-10975) MongoBlobGCTest.consistencyCheckWithRenegadeDelete flaky
Stefan Egli created OAK-10975:
-
Summary: MongoBlobGCTest.consistencyCheckWithRenegadeDelete flaky
Key: OAK-10975
URL: https://issues.apache.org/jira/browse/OAK-10975
Project: Jackrabbit Oak
Issue Type: Task
Components: documentmk
Reporter: Stefan Egli
Below flaky test failure was seen:
{noformat}
java.lang.AssertionError: expected:<1> but was:<2>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:633)
at
org.apache.jackrabbit.oak.plugins.document.MongoBlobGCTest.consistencyCheckWithRenegadeDelete(MongoBlobGCTest.java:326)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10974) testDeletedPropsAndUnmergedBCWithoutCollision flakyness
[
https://issues.apache.org/jira/browse/OAK-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868439#comment-17868439
]
Stefan Egli commented on OAK-10974:
---
PR to disable : https://github.com/apache/jackrabbit-oak/pull/1600
> testDeletedPropsAndUnmergedBCWithoutCollision flakyness
> ---
>
> Key: OAK-10974
> URL: https://issues.apache.org/jira/browse/OAK-10974
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Priority: Minor
>
> Below flaky test failure was seen:
> {noformat}
> [ERROR] Failures:
> [ERROR]
> VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithoutCollision:1738->assertStatsCountsEqual:1333
> ORPHANS_EMPTYPROPS_KEEP_ONE_ALL_PROPS/internalPropRevs e
> xpected:<9> but was:<5>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10869) testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode test failure
[ https://issues.apache.org/jira/browse/OAK-10869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868440#comment-17868440 ] Stefan Egli commented on OAK-10869: --- PR to disable : https://github.com/apache/jackrabbit-oak/pull/1600 > testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode test failure > - > > Key: OAK-10869 > URL: https://issues.apache.org/jira/browse/OAK-10869 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > > https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/org.apache.jackrabbit$oak-store-document/1524/testReport/junit/org.apache.jackrabbit.oak.plugins.document/VersionGarbageCollectorIT/testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode_4__MongoFixture__MongoDB_with_GAP_ORPHANS_EMPTYPROPS_/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10974) testDeletedPropsAndUnmergedBCWithoutCollision flakyness
[
https://issues.apache.org/jira/browse/OAK-10974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli updated OAK-10974:
--
Epic Link: OAK-10739
> testDeletedPropsAndUnmergedBCWithoutCollision flakyness
> ---
>
> Key: OAK-10974
> URL: https://issues.apache.org/jira/browse/OAK-10974
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Priority: Minor
>
> Below flaky test failure was seen:
> {noformat}
> [ERROR] Failures:
> [ERROR]
> VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithoutCollision:1738->assertStatsCountsEqual:1333
> ORPHANS_EMPTYPROPS_KEEP_ONE_ALL_PROPS/internalPropRevs e
> xpected:<9> but was:<5>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OAK-10974) testDeletedPropsAndUnmergedBCWithoutCollision flakyness
Stefan Egli created OAK-10974:
-
Summary: testDeletedPropsAndUnmergedBCWithoutCollision flakyness
Key: OAK-10974
URL: https://issues.apache.org/jira/browse/OAK-10974
Project: Jackrabbit Oak
Issue Type: Task
Components: documentmk
Reporter: Stefan Egli
Below flaky test failure was seen:
{noformat}
[ERROR] Failures:
[ERROR]
VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithoutCollision:1738->assertStatsCountsEqual:1333
ORPHANS_EMPTYPROPS_KEEP_ONE_ALL_PROPS/internalPropRevs e
xpected:<9> but was:<5>
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (OAK-10803) Compress in-memory property values
[ https://issues.apache.org/jira/browse/OAK-10803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli resolved OAK-10803. --- Fix Version/s: 1.68.0 Resolution: Fixed [PR|https://github.com/apache/jackrabbit-oak/pull/1526] merged, thx [~ionutpirlogea]]! Compression is disabled by default. We have encountered performance issues that we first need to work through before we can enable it. Created OAK-10973 to look into performance. > Compress in-memory property values > -- > > Key: OAK-10803 > URL: https://issues.apache.org/jira/browse/OAK-10803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Daniel Iancu >Priority: Major > Fix For: 1.68.0 > > > Some properties in the repository can be quite large and consume a > considerable amount of memory. E.g. large multi-valued properties, blob IDs > that contain an inlined binary. > With a certain size of the value, it may be beneficial to compress it when > loaded into memory/cache and uncompress it only on access. This shouldn't be > too difficult and can be implemented in a backward compatible way, because it > only affects the in-memory representation and not how data is stored. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (OAK-10803) Compress in-memory property values
[ https://issues.apache.org/jira/browse/OAK-10803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17868405#comment-17868405 ] Stefan Egli edited comment on OAK-10803 at 7/24/24 3:32 PM: [PR|https://github.com/apache/jackrabbit-oak/pull/1526] merged, thx [~ionutpirlogea]! Compression is disabled by default. We have encountered performance issues that we first need to work through before we can enable it. Created OAK-10973 to look into performance. was (Author: egli): [PR|https://github.com/apache/jackrabbit-oak/pull/1526] merged, thx [~ionutpirlogea]]! Compression is disabled by default. We have encountered performance issues that we first need to work through before we can enable it. Created OAK-10973 to look into performance. > Compress in-memory property values > -- > > Key: OAK-10803 > URL: https://issues.apache.org/jira/browse/OAK-10803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Daniel Iancu >Priority: Major > Fix For: 1.68.0 > > > Some properties in the repository can be quite large and consume a > considerable amount of memory. E.g. large multi-valued properties, blob IDs > that contain an inlined binary. > With a certain size of the value, it may be beneficial to compress it when > loaded into memory/cache and uncompress it only on access. This shouldn't be > too difficult and can be implemented in a backward compatible way, because it > only affects the in-memory representation and not how data is stored. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10881) Expand oak/docs/participating with some more guidelines
[ https://issues.apache.org/jira/browse/OAK-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17866675#comment-17866675 ] Stefan Egli commented on OAK-10881: --- If there are already known further things to add I'd say yes - otherwise we can always create an ad-hoc ticket when new things pop up? > Expand oak/docs/participating with some more guidelines > --- > > Key: OAK-10881 > URL: https://issues.apache.org/jira/browse/OAK-10881 > Project: Jackrabbit Oak > Issue Type: Documentation > Components: doc >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Fix For: 1.66.0 > > > Proposal: > Tickets/PRs/Workflow: > - Always reference an Oak ticket for each commit/PR (this should include the > jira id in the correct format, eg OAK-10881 instead of Oak 10881) > - minimize PRs; do not modify whitespace/coding style except where needed > - structure tickets/PRs so that things that can be separated are (that can be > useful for backports) > - have test cases (when there's no immediate fix, create a ticket and a PR > just for the test and mark it "ignored", pointing to the actual issue) > - trunk: when done with a ticket, set it to "resolved" and set "Fix Version" > to the next unreleased version > - maintenance branch: re-use the existing Jira ticket and just add to "Fix > Version" (unless the backport is complex) > - avoid committing unfinished stuff to trunk; in particular when a release is > approaching > - add affects-version and fix-version as and when applicable > - PRs that contain multiple commits in general should be "squashed and merged" > Coding Style: > - no wildcard imports > - in general be consistent with the style of the code being modified > - avoid TABs and trailing whitespace -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OAK-10935) flaky testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode
Stefan Egli created OAK-10935:
-
Summary: flaky
testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode
Key: OAK-10935
URL: https://issues.apache.org/jira/browse/OAK-10935
Project: Jackrabbit Oak
Issue Type: Task
Components: documentmk
Reporter: Stefan Egli
There was a flaky test failure [in this jenkins run (no longer
available)|https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/org.apache.jackrabbit$oak-store-document/1570/testReport/org.apache.jackrabbit.oak.plugins.document/VersionGarbageCollectorIT/testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode_0__MemoryFixture__Memory_with_NONE_/]
that reported below.
The unexpected thing is, the test does 4 unmerged and 2 merged BCs. The
resulting doc as seen below however, only contains 4 BCs - 2 unmerged and 2
merged. They are OK not to be cleaned up, that's the expectation due to dryRun.
And the preceding stats must have confirmed no deletion happened in gc().
Except, why are 2 unmerged BCs missing in the output, that's unexpected.
{noformat}
Regression
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode[0:
MemoryFixture: Memory with NONE]
Failing for the past 1 build (Since
#1570 )
Took 16 ms.
Error Message
document not cleaned up:
"_children":true,"_bc":{"r190726417f7-0-1":"true","r190726417f2-0-1":"true","r190726417eb-0-1":"true","r190726417d1-0-1":"true"},"_deleted":{"r19072641788-0-1":"false"},"_collisions":{},"_commitRoot":{},"_sweepRev":{"r0-0-1":"r190726417fd-0-1"},"_revisions":{"r190726417f7-0-1":"c-r190726417fd-0-1","r190726417f2-0-1":"br190726417d9-0-1","r190726417eb-0-1":"br190726417d9-0-1","r190726417d1-0-1":"c-r190726417d9-0-1","r19072641799-0-1":"c","r19072641788-0-1":"c"},"_id":"0:/","_modified":1719906080,"_lastRev":{"r0-0-1":"r190726417fd-0-1"}
Stacktrace
java.lang.AssertionError: document not cleaned up:
"_children":true,"_bc":{"r190726417f7-0-1":"true","r190726417f2-0-1":"true","r190726417eb-0-1":"true","r190726417d1-0-1":"true"},"_deleted":{"r19072641788-0-1":"false"},"_collisions":{},"_commitRoot":{},"_sweepRev":{"r0-0-1":"r190726417fd-0-1"},"_revisions":{"r190726417f7-0-1":"c-r190726417fd-0-1","r190726417f2-0-1":"br190726417d9-0-1","r190726417eb-0-1":"br190726417d9-0-1","r190726417d1-0-1":"c-r190726417d9-0-1","r19072641799-0-1":"c","r19072641788-0-1":"c"},"_id":"0:/","_modified":1719906080,"_lastRev":{"r0-0-1":"r190726417fd-0-1"}
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at
org.apache.jackrabbit.oak.plugins.document.FullGCHelper.assertBranchRevisionNotRemovedFromAllDocuments(FullGCHelper.java:122)
at
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode(VersionGarbageCollectorIT.java:2403)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (OAK-10933) Avoid log about calculating fullGCTimestamp when fullGC is disabled
[
https://issues.apache.org/jira/browse/OAK-10933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli resolved OAK-10933.
---
Fix Version/s: 1.66.0
Resolution: Fixed
PR merged, marking ticket done.
> Avoid log about calculating fullGCTimestamp when fullGC is disabled
> ---
>
> Key: OAK-10933
> URL: https://issues.apache.org/jira/browse/OAK-10933
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Minor
> Fix For: 1.66.0
>
>
> With fullgc disabled below info message were seen - they should be avoided :
> {noformat}
> 01.02.2013 10:11:12.014 *INFO*
> [sling-oak-1-org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService$RevisionGCJob]
> org.apache.jackrabbit.oak.plugins.document.VersionGCRecommendations No
> fullGCTimestamp found, querying for the oldest modified candidate
> 01.02.2013 10:11:12.015 *INFO*
> [sling-oak-1-org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService$RevisionGCJob]
> org.apache.jackrabbit.oak.plugins.document.VersionGCRecommendations
> fullGCTimestamp found: 2004-01-02 23:24:25.026
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10933) Avoid log about calculating fullGCTimestamp when fullGC is disabled
[
https://issues.apache.org/jira/browse/OAK-10933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17862841#comment-17862841
]
Stefan Egli commented on OAK-10933:
---
PR created : https://github.com/apache/jackrabbit-oak/pull/1570
> Avoid log about calculating fullGCTimestamp when fullGC is disabled
> ---
>
> Key: OAK-10933
> URL: https://issues.apache.org/jira/browse/OAK-10933
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Minor
>
> With fullgc disabled below info message were seen - they should be avoided :
> {noformat}
> 01.02.2013 10:11:12.014 *INFO*
> [sling-oak-1-org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService$RevisionGCJob]
> org.apache.jackrabbit.oak.plugins.document.VersionGCRecommendations No
> fullGCTimestamp found, querying for the oldest modified candidate
> 01.02.2013 10:11:12.015 *INFO*
> [sling-oak-1-org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService$RevisionGCJob]
> org.apache.jackrabbit.oak.plugins.document.VersionGCRecommendations
> fullGCTimestamp found: 2004-01-02 23:24:25.026
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OAK-10933) Avoid log about calculating fullGCTimestamp when fullGC is disabled
Stefan Egli created OAK-10933:
-
Summary: Avoid log about calculating fullGCTimestamp when fullGC
is disabled
Key: OAK-10933
URL: https://issues.apache.org/jira/browse/OAK-10933
Project: Jackrabbit Oak
Issue Type: Task
Components: documentmk
Reporter: Stefan Egli
Assignee: Stefan Egli
With fullgc disabled below info message were seen - they should be avoided :
{noformat}
01.02.2013 10:11:12.014 *INFO*
[sling-oak-1-org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService$RevisionGCJob]
org.apache.jackrabbit.oak.plugins.document.VersionGCRecommendations No
fullGCTimestamp found, querying for the oldest modified candidate
01.02.2013 10:11:12.015 *INFO*
[sling-oak-1-org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService$RevisionGCJob]
org.apache.jackrabbit.oak.plugins.document.VersionGCRecommendations
fullGCTimestamp found: 2004-01-02 23:24:25.026
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10843) Flaky fullgc tests
[ https://issues.apache.org/jira/browse/OAK-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17862769#comment-17862769 ] Stefan Egli commented on OAK-10843: --- if I may : another one : https://github.com/apache/jackrabbit-oak/pull/1568 > Flaky fullgc tests > -- > > Key: OAK-10843 > URL: https://issues.apache.org/jira/browse/OAK-10843 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > Fix For: 1.66.0 > > > As noted in OAK-10739 there is potential flakyness in tests with fullgc modes > "BETWEEN_CHECKPOINTS" and others. This ticket is to look into fixing it. > First measure could be to disable them until we have a fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10843) Flaky fullgc tests
[ https://issues.apache.org/jira/browse/OAK-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17861354#comment-17861354 ] Stefan Egli commented on OAK-10843: --- +1, if more flakyness shows up we can create a new ticket > Flaky fullgc tests > -- > > Key: OAK-10843 > URL: https://issues.apache.org/jira/browse/OAK-10843 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > Fix For: 1.66.0 > > > As noted in OAK-10739 there is potential flakyness in tests with fullgc modes > "BETWEEN_CHECKPOINTS" and others. This ticket is to look into fixing it. > First measure could be to disable them until we have a fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (OAK-10914) fullGC exclude paths should use _modified_id index
[
https://issues.apache.org/jira/browse/OAK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli resolved OAK-10914.
---
Fix Version/s: 1.66.0
Resolution: Fixed
> fullGC exclude paths should use _modified_id index
> --
>
> Key: OAK-10914
> URL: https://issues.apache.org/jira/browse/OAK-10914
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Fix For: 1.66.0
>
>
> As
> [noticed|https://github.com/apache/jackrabbit-oak/pull/1547#discussion_r1648725361]
> by [~nuno.santos] the current state of how exclude paths are implemented
> would result in {{_modified_id}} index not being used and thus be slow. This
> needs to be improved.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10914) fullGC exclude paths should use _modified_id index
[
https://issues.apache.org/jira/browse/OAK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17861333#comment-17861333
]
Stefan Egli commented on OAK-10914:
---
[PR|https://github.com/apache/jackrabbit-oak/pull/1563] is merged, created
OAK-10926.to add more tests, marking this ticket here resolved.
> fullGC exclude paths should use _modified_id index
> --
>
> Key: OAK-10914
> URL: https://issues.apache.org/jira/browse/OAK-10914
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> As
> [noticed|https://github.com/apache/jackrabbit-oak/pull/1547#discussion_r1648725361]
> by [~nuno.santos] the current state of how exclude paths are implemented
> would result in {{_modified_id}} index not being used and thus be slow. This
> needs to be improved.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OAK-10926) Add more tests for : fullGC exclude paths should use _modified_id index
Stefan Egli created OAK-10926: - Summary: Add more tests for : fullGC exclude paths should use _modified_id index Key: OAK-10926 URL: https://issues.apache.org/jira/browse/OAK-10926 Project: Jackrabbit Oak Issue Type: Task Components: documentmk Reporter: Stefan Egli OAK-10914 ensured exclude paths used the modified-id index. This ticket here is to add more testing in that area still. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (OAK-10926) Add more tests for : fullGC exclude paths should use _modified_id index
[ https://issues.apache.org/jira/browse/OAK-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli reassigned OAK-10926: - Assignee: Stefan Egli > Add more tests for : fullGC exclude paths should use _modified_id index > --- > > Key: OAK-10926 > URL: https://issues.apache.org/jira/browse/OAK-10926 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > > OAK-10914 ensured exclude paths used the modified-id index. This ticket here > is to add more testing in that area still. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10914) fullGC exclude paths should use _modified_id index
[
https://issues.apache.org/jira/browse/OAK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860501#comment-17860501
]
Stefan Egli commented on OAK-10914:
---
Thx [~nuno.santos] for the examples, makes sense!
I have now switched negation to negative lookahead in [this
PR|https://github.com/apache/jackrabbit-oak/pull/1563] (should still add a unit
test for this as a follow-up PR)
> fullGC exclude paths should use _modified_id index
> --
>
> Key: OAK-10914
> URL: https://issues.apache.org/jira/browse/OAK-10914
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> As
> [noticed|https://github.com/apache/jackrabbit-oak/pull/1547#discussion_r1648725361]
> by [~nuno.santos] the current state of how exclude paths are implemented
> would result in {{_modified_id}} index not being used and thus be slow. This
> needs to be improved.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10921) Fix race condition while resetting fullGC variables from oak-run
[ https://issues.apache.org/jira/browse/OAK-10921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860406#comment-17860406 ] Stefan Egli commented on OAK-10921: --- Option 2 looks easier to me > Fix race condition while resetting fullGC variables from oak-run > > > Key: OAK-10921 > URL: https://issues.apache.org/jira/browse/OAK-10921 > Project: Jackrabbit Oak > Issue Type: Task >Reporter: Rishabh Daim >Assignee: Rishabh Daim >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (OAK-10918) Embed and log an explain for the fullgc mongo query
[
https://issues.apache.org/jira/browse/OAK-10918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli resolved OAK-10918.
---
Fix Version/s: 1.66.0
Resolution: Fixed
[PR|https://github.com/apache/jackrabbit-oak/pull/1559] merged, marking ticket
done.
> Embed and log an explain for the fullgc mongo query
> ---
>
> Key: OAK-10918
> URL: https://issues.apache.org/jira/browse/OAK-10918
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
> Fix For: 1.66.0
>
>
> We should do an {{explain}} of the query that is ultimately used in
> MongoGCSupport for fullgc, and log (at INFO) the most relevant pieces of the
> winning plan (or just the entire winning plan). Doing this once per reconfig
> (which is probably currently once per JVM-incarnation) shouldn't be intrusive
> - but it would make it observable which, if any, index is being used. Eg it
> should say something like
> {noformat}
> "stage": "IXSCAN",
> "indexName": "_modified_1__id_1",
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10914) fullGC exclude paths should use _modified_id index
[
https://issues.apache.org/jira/browse/OAK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860202#comment-17860202
]
Stefan Egli commented on OAK-10914:
---
[~nuno.santos], I'm suspecting some other condition that causes it to use the
id index and not modified-id. It seems independent of how the exclude path
condition is created - it's always preferring the id index.
Actually, the modified-id index is created non-sparse, so it should contain all
documents - besides id is mandatory anyway. So it seems difficult to serve as
explanation why it doesn't prefer that index?
When I do give it a hint, it does use it just fine though (when without hint it
never chooses it). So it could be some artifact of how the query exactly looks,
or something else... but for me I can't base it on the "not vs
negative-look-ahead" part ..
> fullGC exclude paths should use _modified_id index
> --
>
> Key: OAK-10914
> URL: https://issues.apache.org/jira/browse/OAK-10914
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> As
> [noticed|https://github.com/apache/jackrabbit-oak/pull/1547#discussion_r1648725361]
> by [~nuno.santos] the current state of how exclude paths are implemented
> would result in {{_modified_id}} index not being used and thus be slow. This
> needs to be improved.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10918) Embed and log an explain for the fullgc mongo query
[
https://issues.apache.org/jira/browse/OAK-10918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17860167#comment-17860167
]
Stefan Egli commented on OAK-10918:
---
[PR|https://github.com/apache/jackrabbit-oak/pull/1559] created
> Embed and log an explain for the fullgc mongo query
> ---
>
> Key: OAK-10918
> URL: https://issues.apache.org/jira/browse/OAK-10918
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> We should do an {{explain}} of the query that is ultimately used in
> MongoGCSupport for fullgc, and log (at INFO) the most relevant pieces of the
> winning plan (or just the entire winning plan). Doing this once per reconfig
> (which is probably currently once per JVM-incarnation) shouldn't be intrusive
> - but it would make it observable which, if any, index is being used. Eg it
> should say something like
> {noformat}
> "stage": "IXSCAN",
> "indexName": "_modified_1__id_1",
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (OAK-10896) Add osgi config for removal of deleted properties and orphaned nodes
[ https://issues.apache.org/jira/browse/OAK-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10896: -- Fix Version/s: 1.66.0 > Add osgi config for removal of deleted properties and orphaned nodes > > > Key: OAK-10896 > URL: https://issues.apache.org/jira/browse/OAK-10896 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Horia Poradici >Priority: Major > Fix For: 1.66.0 > > > OSGI configuration options should be added for full garbage collection > options GAP_ORPHANS and EMPTY_PROPERTIES. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (OAK-10896) Add osgi config for removal of deleted properties and orphaned nodes
[ https://issues.apache.org/jira/browse/OAK-10896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli resolved OAK-10896. --- Assignee: (was: Rishabh Daim) Resolution: Fixed [PR|https://github.com/apache/jackrabbit-oak/pull/1539] merged, thanks, marking ticket as resolved > Add osgi config for removal of deleted properties and orphaned nodes > > > Key: OAK-10896 > URL: https://issues.apache.org/jira/browse/OAK-10896 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Horia Poradici >Priority: Major > Fix For: 1.66.0 > > > OSGI configuration options should be added for full garbage collection > options GAP_ORPHANS and EMPTY_PROPERTIES. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10881) Expand oak/docs/participating with some more guidelines
[
https://issues.apache.org/jira/browse/OAK-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17859894#comment-17859894
]
Stefan Egli commented on OAK-10881:
---
+1 to avoiding static imports of methods - eg {{Set.of()}} is much more
readable than just {{of()}} - also see [the inventor's opinion on
it|https://docs.oracle.com/javase/8/docs/technotes/guides/language/static-import.html]
{quote}So when should you use static import? Very sparingly! Only use it when
you'd otherwise be tempted to declare local copies of constants, or to abuse
inheritance (the Constant Interface Antipattern). In other words, use it when
you require frequent access to static members from one or two classes. If you
overuse the static import feature, it can make your program unreadable and
unmaintainable, polluting its namespace with all the static members you import.
Readers of your code (including you, a few months after you wrote it) will not
know which class a static member comes from. Importing all of the static
members from a class can be particularly harmful to readability; if you need
only one or two members, import them individually. Used appropriately, static
import can make your program more readable, by removing the boilerplate of
repetition of class names. {quote}
> Expand oak/docs/participating with some more guidelines
> ---
>
> Key: OAK-10881
> URL: https://issues.apache.org/jira/browse/OAK-10881
> Project: Jackrabbit Oak
> Issue Type: Documentation
> Components: doc
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Fix For: 1.66.0
>
>
> Proposal:
> Tickets/PRs/Workflow:
> - Always reference an Oak ticket for each commit/PR (this should include the
> jira id in the correct format, eg OAK-10881 instead of Oak 10881)
> - minimize PRs; do not modify whitespace/coding style except where needed
> - structure tickets/PRs so that things that can be separated are (that can be
> useful for backports)
> - have test cases (when there's no immediate fix, create a ticket and a PR
> just for the test and mark it "ignored", pointing to the actual issue)
> - trunk: when done with a ticket, set it to "resolved" and set "Fix Version"
> to the next unreleased version
> - maintenance branch: re-use the existing Jira ticket and just add to "Fix
> Version" (unless the backport is complex)
> - avoid committing unfinished stuff to trunk; in particular when a release is
> approaching
> - add affects-version and fix-version as and when applicable
> - PRs that contain multiple commits in general should be "squashed and merged"
> Coding Style:
> - no wildcard imports
> - in general be consistent with the style of the code being modified
> - avoid TABs and trailing whitespace
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (OAK-10918) Embed and log an explain for the fullgc mongo query
[
https://issues.apache.org/jira/browse/OAK-10918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli updated OAK-10918:
--
Epic Link: OAK-10739
> Embed and log an explain for the fullgc mongo query
> ---
>
> Key: OAK-10918
> URL: https://issues.apache.org/jira/browse/OAK-10918
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> We should do an {{explain}} of the query that is ultimately used in
> MongoGCSupport for fullgc, and log (at INFO) the most relevant pieces of the
> winning plan (or just the entire winning plan). Doing this once per reconfig
> (which is probably currently once per JVM-incarnation) shouldn't be intrusive
> - but it would make it observable which, if any, index is being used. Eg it
> should say something like
> {noformat}
> "stage": "IXSCAN",
> "indexName": "_modified_1__id_1",
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10914) fullGC exclude paths should use _modified_id index
[
https://issues.apache.org/jira/browse/OAK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17859740#comment-17859740
]
Stefan Egli commented on OAK-10914:
---
Looks like specifying a hint for "_modified_1__id_1" could achieve this.
Created OAK-10918 to add a log.info (once per reconfig/startup) with the actual
winning plan obtained via explain. Having that would allow to verify
introducing the index hint achieves what we want.
> fullGC exclude paths should use _modified_id index
> --
>
> Key: OAK-10914
> URL: https://issues.apache.org/jira/browse/OAK-10914
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> As
> [noticed|https://github.com/apache/jackrabbit-oak/pull/1547#discussion_r1648725361]
> by [~nuno.santos] the current state of how exclude paths are implemented
> would result in {{_modified_id}} index not being used and thus be slow. This
> needs to be improved.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OAK-10918) Embed and log an explain for the fullgc mongo query
Stefan Egli created OAK-10918:
-
Summary: Embed and log an explain for the fullgc mongo query
Key: OAK-10918
URL: https://issues.apache.org/jira/browse/OAK-10918
Project: Jackrabbit Oak
Issue Type: Task
Components: documentmk
Reporter: Stefan Egli
Assignee: Stefan Egli
We should do an {{explain}} of the query that is ultimately used in
MongoGCSupport for fullgc, and log (at INFO) the most relevant pieces of the
winning plan (or just the entire winning plan). Doing this once per reconfig
(which is probably currently once per JVM-incarnation) shouldn't be intrusive -
but it would make it observable which, if any, index is being used. Eg it
should say something like
{noformat}
"stage": "IXSCAN",
"indexName": "_modified_1__id_1",
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10843) Flaky fullgc tests
[ https://issues.apache.org/jira/browse/OAK-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17859679#comment-17859679 ] Stefan Egli commented on OAK-10843: --- another PR for a flaky run -> https://github.com/apache/jackrabbit-oak/pull/1554 > Flaky fullgc tests > -- > > Key: OAK-10843 > URL: https://issues.apache.org/jira/browse/OAK-10843 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > As noted in OAK-10739 there is potential flakyness in tests with fullgc modes > "BETWEEN_CHECKPOINTS" and others. This ticket is to look into fixing it. > First measure could be to disable them until we have a fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10914) fullGC exclude paths should use _modified_id index
[
https://issues.apache.org/jira/browse/OAK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17859643#comment-17859643
]
Stefan Egli commented on OAK-10914:
---
* [PR|https://github.com/apache/jackrabbit-oak/pull/1553] to disable
mongo-server-side exclude filtering until the performance aspect is fixed. The
result is that excludes are applied client-side (so still not great performance
but at least it doesn't do a full repository scan - plus performance depends on
how include paths are setup)
next step:
* improve performance by optimizing query
> fullGC exclude paths should use _modified_id index
> --
>
> Key: OAK-10914
> URL: https://issues.apache.org/jira/browse/OAK-10914
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> As
> [noticed|https://github.com/apache/jackrabbit-oak/pull/1547#discussion_r1648725361]
> by [~nuno.santos] the current state of how exclude paths are implemented
> would result in {{_modified_id}} index not being used and thus be slow. This
> needs to be improved.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (OAK-10914) fullGC exclude paths should use _modified_id index
[
https://issues.apache.org/jira/browse/OAK-10914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli updated OAK-10914:
--
Epic Link: OAK-10739
> fullGC exclude paths should use _modified_id index
> --
>
> Key: OAK-10914
> URL: https://issues.apache.org/jira/browse/OAK-10914
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Major
>
> As
> [noticed|https://github.com/apache/jackrabbit-oak/pull/1547#discussion_r1648725361]
> by [~nuno.santos] the current state of how exclude paths are implemented
> would result in {{_modified_id}} index not being used and thus be slow. This
> needs to be improved.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OAK-10914) fullGC exclude paths should use _modified_id index
Stefan Egli created OAK-10914:
-
Summary: fullGC exclude paths should use _modified_id index
Key: OAK-10914
URL: https://issues.apache.org/jira/browse/OAK-10914
Project: Jackrabbit Oak
Issue Type: Task
Components: documentmk
Reporter: Stefan Egli
Assignee: Stefan Egli
As
[noticed|https://github.com/apache/jackrabbit-oak/pull/1547#discussion_r1648725361]
by [~nuno.santos] the current state of how exclude paths are implemented would
result in {{_modified_id}} index not being used and thus be slow. This needs to
be improved.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (OAK-10742) Introduce include/exclude lists for detailedGC
[ https://issues.apache.org/jira/browse/OAK-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17856537#comment-17856537 ] Stefan Egli edited comment on OAK-10742 at 6/20/24 4:41 PM: [PR|https://github.com/apache/jackrabbit-oak/pull/1547] merged. For the record : long paths are currently not included when include paths are configured - they are only included if no include path is set (reason is to avoid impact on performance due to missing index with "_path") - they are however excluded as well (just only client-side and not via the mongo query) was (Author: egli): [PR|https://github.com/apache/jackrabbit-oak/pull/1547] merged. For the record : long paths are currently not included when include paths are configured - they are only included if no include path is set - they are however excluded as well (just only client-side and not via the mongo query) > Introduce include/exclude lists for detailedGC > -- > > Key: OAK-10742 > URL: https://issues.apache.org/jira/browse/OAK-10742 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > Labels: DetailedGC > Fix For: 1.66.0 > > > We should have code that allows to specify: > * an exclude list of path prefixes that are ignored for detailed gc in > general. one example for this could be to not gc > /jcr:system/jcr:versionStorage at all. This option should apply to all > different detailedGC modes > * an option for an include list of path prefixes that should only be > considered for gc. This option is for example useful for testing > deleteEmptyProperties. > Note that the initial description was suggesting wildcards - that seems > unnecessarily complicated though, and just having path prefixes seems to be > much easier. So the description now assumes path prefixes rather than > wildcards. > This ticket here covers the implementation of these lists - the configuration > of this is handled in a separate ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (OAK-10742) Introduce include/exclude lists for detailedGC
[ https://issues.apache.org/jira/browse/OAK-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17856537#comment-17856537 ] Stefan Egli edited comment on OAK-10742 at 6/20/24 4:40 PM: [PR|https://github.com/apache/jackrabbit-oak/pull/1547] merged. For the record : long paths are currently not included when include paths are configured - they are only included if no include path is set - they are however excluded as well (just only client-side and not via the mongo query) was (Author: egli): [PR|https://github.com/apache/jackrabbit-oak/pull/1547] merged. For the record : long paths are currently not include when include paths are configured - they are only included if no include path is set - they are however excluded as well (just only client-side and not via the mongo query) > Introduce include/exclude lists for detailedGC > -- > > Key: OAK-10742 > URL: https://issues.apache.org/jira/browse/OAK-10742 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > Labels: DetailedGC > Fix For: 1.66.0 > > > We should have code that allows to specify: > * an exclude list of path prefixes that are ignored for detailed gc in > general. one example for this could be to not gc > /jcr:system/jcr:versionStorage at all. This option should apply to all > different detailedGC modes > * an option for an include list of path prefixes that should only be > considered for gc. This option is for example useful for testing > deleteEmptyProperties. > Note that the initial description was suggesting wildcards - that seems > unnecessarily complicated though, and just having path prefixes seems to be > much easier. So the description now assumes path prefixes rather than > wildcards. > This ticket here covers the implementation of these lists - the configuration > of this is handled in a separate ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10742) Introduce include/exclude lists for detailedGC
[ https://issues.apache.org/jira/browse/OAK-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17856537#comment-17856537 ] Stefan Egli commented on OAK-10742: --- [PR|https://github.com/apache/jackrabbit-oak/pull/1547] merged. For the record : long paths are currently not include when include paths are configured - they are only included if no include path is set - they are however excluded as well (just only client-side and not via the mongo query) > Introduce include/exclude lists for detailedGC > -- > > Key: OAK-10742 > URL: https://issues.apache.org/jira/browse/OAK-10742 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > Labels: DetailedGC > Fix For: 1.66.0 > > > We should have code that allows to specify: > * an exclude list of path prefixes that are ignored for detailed gc in > general. one example for this could be to not gc > /jcr:system/jcr:versionStorage at all. This option should apply to all > different detailedGC modes > * an option for an include list of path prefixes that should only be > considered for gc. This option is for example useful for testing > deleteEmptyProperties. > Note that the initial description was suggesting wildcards - that seems > unnecessarily complicated though, and just having path prefixes seems to be > much easier. So the description now assumes path prefixes rather than > wildcards. > This ticket here covers the implementation of these lists - the configuration > of this is handled in a separate ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (OAK-10742) Introduce include/exclude lists for detailedGC
[ https://issues.apache.org/jira/browse/OAK-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli resolved OAK-10742. --- Fix Version/s: 1.66.0 Resolution: Fixed > Introduce include/exclude lists for detailedGC > -- > > Key: OAK-10742 > URL: https://issues.apache.org/jira/browse/OAK-10742 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > Labels: DetailedGC > Fix For: 1.66.0 > > > We should have code that allows to specify: > * an exclude list of path prefixes that are ignored for detailed gc in > general. one example for this could be to not gc > /jcr:system/jcr:versionStorage at all. This option should apply to all > different detailedGC modes > * an option for an include list of path prefixes that should only be > considered for gc. This option is for example useful for testing > deleteEmptyProperties. > Note that the initial description was suggesting wildcards - that seems > unnecessarily complicated though, and just having path prefixes seems to be > much easier. So the description now assumes path prefixes rather than > wildcards. > This ticket here covers the implementation of these lists - the configuration > of this is handled in a separate ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10906) sonar builds fail with ' An API incompatibility was encountered', 'UnsupportedClassVersionError'
[
https://issues.apache.org/jira/browse/OAK-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17856456#comment-17856456
]
Stefan Egli commented on OAK-10906:
---
This is confusing - it talks about a SonarQube version 8 while we should
probably be at 9 at least
{noformat}
2024-06-19T16:49:58.7850566Z [INFO] SonarQube version: 8.0.0.55282
{noformat}
Otherwise yes, it uses JDK 11
{noformat}
2024-06-19T16:01:16.0176940Z JAVA_HOME:
/opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/11.0.23-9/x64
{noformat}
The actual exception states that some sonar class is JDK 17
{noformat}
while executing
org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar:
java.lang.UnsupportedClassVersionError:
org/sonar/batch/bootstrapper/EnvironmentInformation has been compiled by a more
recent version of the Java Runtime (class file version 61.0)
{noformat}
> sonar builds fail with ' An API incompatibility was encountered',
> 'UnsupportedClassVersionError'
>
>
> Key: OAK-10906
> URL: https://issues.apache.org/jira/browse/OAK-10906
> Project: Jackrabbit Oak
> Issue Type: Task
>Reporter: Stefan Egli
>Priority: Major
>
> This has happened repeatedly recently - latest example
> https://github.com/apache/jackrabbit-oak/actions/runs/9585078456/job/26430154398?pr=1548
> {noformat}
> Error: Failed to execute goal
> org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar
> (default-cli) on project jackrabbit-oak: Execution default-cli of goal
> org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar failed: An
> API incompatibility was encountered while executing
> org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar:
> java.lang.UnsupportedClassVersionError:
> org/sonar/batch/bootstrapper/EnvironmentInformation has been compiled by a
> more recent version of the Java Runtime (class file version 61.0), this
> version of the Java Runtime only recognizes class file versions up to 55.0
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10881) Expand oak/docs/participating with some more guidelines
[ https://issues.apache.org/jira/browse/OAK-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17856435#comment-17856435 ] Stefan Egli commented on OAK-10881: --- +1, eg like "consider discussing this in oak-dev list" depending on the case? > Expand oak/docs/participating with some more guidelines > --- > > Key: OAK-10881 > URL: https://issues.apache.org/jira/browse/OAK-10881 > Project: Jackrabbit Oak > Issue Type: Documentation > Components: doc >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > > Proposal: > Tickets/PRs/Workflow: > - Always reference an Oak ticket for each commit/PR (this should include the > jira id in the correct format, eg OAK-10881 instead of Oak 10881) > - minimize PRs; do not modify whitespace/coding style except where needed > - structure tickets/PRs so that things that can be separated are (that can be > useful for backports) > - have test cases (when there's no immediate fix, create a ticket and a PR > just for the test and mark it "ignored", pointing to the actual issue) > - trunk: when done with a ticket, set it to "resolved" and set "Fix Version" > to the next unreleased version > - maintenance branch: re-use the existing Jira ticket and just add to "Fix > Version" (unless the backport is complex) > - avoid committing unfinished stuff to trunk; in particular when a release is > approaching > - add affects-version and fix-version as and when applicable > - PRs that contain multiple commits in general should be "squashed and merged" > Coding Style: > - no wildcard imports > - in general be consistent with the style of the code being modified > - avoid TABs and trailing whitespace -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10742) Introduce include/exclude lists for detailedGC
[ https://issues.apache.org/jira/browse/OAK-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17856434#comment-17856434 ] Stefan Egli commented on OAK-10742: --- Exactly, FullGC is a part of VersionGC, from how it is executed, so it does run regularly. The idea of excludes is for example to allow for production tests on a subset of paths which can be fine-controlled this way (with having both include and exclude options). Or for example one might want to opt to having FullGC not touch /jcr:system/jcr:versionStorage for some reason, and that could be achieved via an exclude path > Introduce include/exclude lists for detailedGC > -- > > Key: OAK-10742 > URL: https://issues.apache.org/jira/browse/OAK-10742 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > Labels: DetailedGC > > We should have code that allows to specify: > * an exclude list of path prefixes that are ignored for detailed gc in > general. one example for this could be to not gc > /jcr:system/jcr:versionStorage at all. This option should apply to all > different detailedGC modes > * an option for an include list of path prefixes that should only be > considered for gc. This option is for example useful for testing > deleteEmptyProperties. > Note that the initial description was suggesting wildcards - that seems > unnecessarily complicated though, and just having path prefixes seems to be > much easier. So the description now assumes path prefixes rather than > wildcards. > This ticket here covers the implementation of these lists - the configuration > of this is handled in a separate ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10906) sonar builds fail with ' An API incompatibility was encountered', 'UnsupportedClassVersionError'
[
https://issues.apache.org/jira/browse/OAK-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli updated OAK-10906:
--
Summary: sonar builds fail with ' An API incompatibility was encountered',
'UnsupportedClassVersionError' (was: sonar builds fail with ' An API
incompatibility was encountered')
> sonar builds fail with ' An API incompatibility was encountered',
> 'UnsupportedClassVersionError'
>
>
> Key: OAK-10906
> URL: https://issues.apache.org/jira/browse/OAK-10906
> Project: Jackrabbit Oak
> Issue Type: Task
>Reporter: Stefan Egli
>Priority: Major
>
> This has happened repeatedly recently - latest example
> https://github.com/apache/jackrabbit-oak/actions/runs/9585078456/job/26430154398?pr=1548
> {noformat}
> Error: Failed to execute goal
> org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar
> (default-cli) on project jackrabbit-oak: Execution default-cli of goal
> org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar failed: An
> API incompatibility was encountered while executing
> org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar:
> java.lang.UnsupportedClassVersionError:
> org/sonar/batch/bootstrapper/EnvironmentInformation has been compiled by a
> more recent version of the Java Runtime (class file version 61.0), this
> version of the Java Runtime only recognizes class file versions up to 55.0
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (OAK-10906) sonar builds fail with ' An API incompatibility was encountered'
[
https://issues.apache.org/jira/browse/OAK-10906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli updated OAK-10906:
--
Summary: sonar builds fail with ' An API incompatibility was encountered'
(was: sonar builds fail with )
> sonar builds fail with ' An API incompatibility was encountered'
>
>
> Key: OAK-10906
> URL: https://issues.apache.org/jira/browse/OAK-10906
> Project: Jackrabbit Oak
> Issue Type: Task
>Reporter: Stefan Egli
>Priority: Major
>
> This has happened repeatedly recently - latest example
> https://github.com/apache/jackrabbit-oak/actions/runs/9585078456/job/26430154398?pr=1548
> {noformat}
> Error: Failed to execute goal
> org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar
> (default-cli) on project jackrabbit-oak: Execution default-cli of goal
> org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar failed: An
> API incompatibility was encountered while executing
> org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar:
> java.lang.UnsupportedClassVersionError:
> org/sonar/batch/bootstrapper/EnvironmentInformation has been compiled by a
> more recent version of the Java Runtime (class file version 61.0), this
> version of the Java Runtime only recognizes class file versions up to 55.0
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (OAK-10907) fix flaky VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode
Stefan Egli created OAK-10907:
-
Summary: fix flaky
VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode
Key: OAK-10907
URL: https://issues.apache.org/jira/browse/OAK-10907
Project: Jackrabbit Oak
Issue Type: Task
Components: documentmk
Reporter: Stefan Egli
Test seems flaky as per
https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/org.apache.jackrabbit$oak-store-document/1546/testReport/junit/org.apache.jackrabbit.oak.plugins.document/VersionGarbageCollectorIT/testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode_2__MongoFixture__MongoDB_with_NONE_/
which reports
{noformat}
Regression
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode[2:
MongoFixture: MongoDB with NONE]
Failing for the past 1 build (Since
#1546 )
Took 0.2 sec.
Error Message
document not cleaned up:
"_children":true,"_bc":{"r1902734d4e4-0-1":"true","r1902734d4df-0-1":"true","r1902734d4d8-0-1":"true","r1902734d4be-0-1":"true"},"_deleted":{"r1902734d473-0-1":"false"},"_collisions":{},"_commitRoot":{},"_sweepRev":{"r0-0-1":"r1902734d4f6-0-1"},"_revisions":{"r1902734d4e4-0-1":"c-r1902734d4f6-0-1","r1902734d4df-0-1":"br1902734d4c6-0-1","r1902734d4d8-0-1":"br1902734d4c6-0-1","r1902734d4be-0-1":"c-r1902734d4c6-0-1","r1902734d484-0-1":"c","r1902734d473-0-1":"c"},"_id":"0:/","_modCount":22,"_modified":1718644690,"_lastRev":{"r0-0-1":"r1902734d4f6-0-1"}
Stacktrace
java.lang.AssertionError: document not cleaned up:
"_children":true,"_bc":{"r1902734d4e4-0-1":"true","r1902734d4df-0-1":"true","r1902734d4d8-0-1":"true","r1902734d4be-0-1":"true"},"_deleted":{"r1902734d473-0-1":"false"},"_collisions":{},"_commitRoot":{},"_sweepRev":{"r0-0-1":"r1902734d4f6-0-1"},"_revisions":{"r1902734d4e4-0-1":"c-r1902734d4f6-0-1","r1902734d4df-0-1":"br1902734d4c6-0-1","r1902734d4d8-0-1":"br1902734d4c6-0-1","r1902734d4be-0-1":"c-r1902734d4c6-0-1","r1902734d484-0-1":"c","r1902734d473-0-1":"c"},"_id":"0:/","_modCount":22,"_modified":1718644690,"_lastRev":{"r0-0-1":"r1902734d4f6-0-1"}
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at
org.apache.jackrabbit.oak.plugins.document.FullGCHelper.assertBranchRevisionNotRemovedFromAllDocuments(FullGCHelper.java:122)
at
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode(VersionGarbageCollectorIT.java:2294)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at
[jira] [Created] (OAK-10906) sonar builds fail with
Stefan Egli created OAK-10906:
-
Summary: sonar builds fail with
Key: OAK-10906
URL: https://issues.apache.org/jira/browse/OAK-10906
Project: Jackrabbit Oak
Issue Type: Task
Reporter: Stefan Egli
This has happened repeatedly recently - latest example
https://github.com/apache/jackrabbit-oak/actions/runs/9585078456/job/26430154398?pr=1548
{noformat}
Error: Failed to execute goal
org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar
(default-cli) on project jackrabbit-oak: Execution default-cli of goal
org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar failed: An
API incompatibility was encountered while executing
org.sonarsource.scanner.maven:sonar-maven-plugin:3.10.0.2594:sonar:
java.lang.UnsupportedClassVersionError:
org/sonar/batch/bootstrapper/EnvironmentInformation has been compiled by a more
recent version of the Java Runtime (class file version 61.0), this version of
the Java Runtime only recognizes class file versions up to 55.0
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (OAK-10845) store-document: occasional test failure: BranchCommitGCTest.unmergedMergedRemoveChild
[ https://issues.apache.org/jira/browse/OAK-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17856298#comment-17856298 ] Stefan Egli edited comment on OAK-10845 at 6/19/24 4:02 PM: More flaky test removals in PR : https://github.com/apache/jackrabbit-oak/pull/1548 this one supersedes the PR#1536 was (Author: egli): More flaky test removals in PR : https://github.com/apache/jackrabbit-oak/pull/1548 > store-document: occasional test failure: > BranchCommitGCTest.unmergedMergedRemoveChild > - > > Key: OAK-10845 > URL: https://issues.apache.org/jira/browse/OAK-10845 > Project: Jackrabbit Oak > Issue Type: Bug > Components: documentmk >Reporter: Julian Reschke >Priority: Minor > > [ERROR] Failures: > [ERROR] BranchCommitGCTest.unmergedMergedRemoveChild:623 > ORPHANS_EMPTYPROPS_BETWEEN_CHECKPOINTS_WITH_UNMERGED_BC/internalProps > expected:<2> but was:<1> -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10845) store-document: occasional test failure: BranchCommitGCTest.unmergedMergedRemoveChild
[ https://issues.apache.org/jira/browse/OAK-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17856298#comment-17856298 ] Stefan Egli commented on OAK-10845: --- More flaky test removals in PR : https://github.com/apache/jackrabbit-oak/pull/1548 > store-document: occasional test failure: > BranchCommitGCTest.unmergedMergedRemoveChild > - > > Key: OAK-10845 > URL: https://issues.apache.org/jira/browse/OAK-10845 > Project: Jackrabbit Oak > Issue Type: Bug > Components: documentmk >Reporter: Julian Reschke >Priority: Minor > > [ERROR] Failures: > [ERROR] BranchCommitGCTest.unmergedMergedRemoveChild:623 > ORPHANS_EMPTYPROPS_BETWEEN_CHECKPOINTS_WITH_UNMERGED_BC/internalProps > expected:<2> but was:<1> -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10742) Introduce include/exclude lists for detailedGC
[ https://issues.apache.org/jira/browse/OAK-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17856293#comment-17856293 ] Stefan Egli commented on OAK-10742: --- PR created at https://github.com/apache/jackrabbit-oak/pull/1547 > Introduce include/exclude lists for detailedGC > -- > > Key: OAK-10742 > URL: https://issues.apache.org/jira/browse/OAK-10742 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > Labels: DetailedGC > > We should have code that allows to specify: > * an exclude list of path prefixes that are ignored for detailed gc in > general. one example for this could be to not gc > /jcr:system/jcr:versionStorage at all. This option should apply to all > different detailedGC modes > * an option for an include list of path prefixes that should only be > considered for gc. This option is for example useful for testing > deleteEmptyProperties. > Note that the initial description was suggesting wildcards - that seems > unnecessarily complicated though, and just having path prefixes seems to be > much easier. So the description now assumes path prefixes rather than > wildcards. > This ticket here covers the implementation of these lists - the configuration > of this is handled in a separate ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10742) Introduce include/exclude lists for detailedGC
[ https://issues.apache.org/jira/browse/OAK-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10742: -- Description: We should have code that allows to specify: * an exclude list of path prefixes that are ignored for detailed gc in general. one example for this could be to not gc /jcr:system/jcr:versionStorage at all. This option should apply to all different detailedGC modes * an option for an include list of path prefixes that should only be considered for gc. This option is for example useful for testing deleteEmptyProperties. Note that the initial description was suggesting wildcards - that seems unnecessarily complicated though, and just having path prefixes seems to be much easier. So the description now assumes path prefixes rather than wildcards. This ticket here covers the implementation of these lists - the configuration of this is handled in a separate ticket. was: We should have config options that define: * an exclude list of wildcard based paths that are ignored for detailed gc in general. one example for this could be to not gc /jcr:system/jcr:versionStorage at all. This option should apply to all different detailedGC modes * a temporary (deprecated) option for an include list of wildcard based paths that should only be considered for gc. This option should apply to deleteEmptyProperties (initially). > Introduce include/exclude lists for detailedGC > -- > > Key: OAK-10742 > URL: https://issues.apache.org/jira/browse/OAK-10742 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > Labels: DetailedGC > > We should have code that allows to specify: > * an exclude list of path prefixes that are ignored for detailed gc in > general. one example for this could be to not gc > /jcr:system/jcr:versionStorage at all. This option should apply to all > different detailedGC modes > * an option for an include list of path prefixes that should only be > considered for gc. This option is for example useful for testing > deleteEmptyProperties. > Note that the initial description was suggesting wildcards - that seems > unnecessarily complicated though, and just having path prefixes seems to be > much easier. So the description now assumes path prefixes rather than > wildcards. > This ticket here covers the implementation of these lists - the configuration > of this is handled in a separate ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (OAK-8848) Impossible to replace a mix:versionable by a different mix:versionable
[
https://issues.apache.org/jira/browse/OAK-8848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Egli resolved OAK-8848.
--
Fix Version/s: 1.66.0
Resolution: Fixed
PR https://github.com/apache/jackrabbit-oak/pull/1474 squash merged, thx
[~horia__poradici]
> Impossible to replace a mix:versionable by a different mix:versionable
> --
>
> Key: OAK-8848
> URL: https://issues.apache.org/jira/browse/OAK-8848
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, jcr
>Affects Versions: 1.16.0
> Environment: The problem occurred from within Sling (Starter 11)
> using oak 1.16.0.
>Reporter: Hans-Peter Stoerr
>Priority: Minor
> Fix For: 1.66.0
>
>
> If you delete a node with mixin mix:versionable and move another node with
> the same mixin in its place (that is, it now has the very same path), you get
> a ConstraintViolationException that complains that you are changing protected
> properties. Thus, this operation is impossible to perform within one
> transaction. But splitting it into two transactions is risky in many cases.
> It seems there is a mechanism that checks for changes of protected
> properties, and ignores that this is now a different node.
> If you have two nodes /somewhere/1 and /somewhere/2 which both are
> nt:unstructured and have mix:versionable, the following code triggers an
> exception but shouldn't:
> session.removeItem("/somewhere/2");
> session.move("/somewhere/1", "/somewhere/2");
> session.save();
> The actual stacktrace thrown is:
> javax.jcr.nodetype.ConstraintViolationException: OakConstraint0100: Property
> is protected: jcr:versionHistory
> at
> org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:226)
> at
> org.apache.jackrabbit.oak.api.CommitFailedException.asRepositoryException(CommitFailedException.java:213)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.newRepositoryException(SessionDelegate.java:669)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.save(SessionDelegate.java:495)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl$8.performVoid(SessionImpl.java:420)
> at
> org.apache.jackrabbit.oak.jcr.delegate.SessionDelegate.performVoid(SessionDelegate.java:273)
> at
> org.apache.jackrabbit.oak.jcr.session.SessionImpl.save(SessionImpl.java:417)
> at
> org.apache.sling.jcr.oak.server.internal.TcclWrappingJackrabbitSession.save(TcclWrappingJackrabbitSession.java:212)
> at
> org.apache.jsp.apps.composum.prototype.assets.pagesintegration.hpsx.hpsx_jsp._jspService(hpsx_jsp.java:115)
> at
> org.apache.sling.scripting.jsp.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
> (irrelevant sling stuff omitted)
> Caused by: org.apache.jackrabbit.oak.api.CommitFailedException:
> OakConstraint0100: Property is protected: jcr:versionHistory
> at
> org.apache.jackrabbit.oak.plugins.version.VersionEditor.throwProtected(VersionEditor.java:257)
> at
> org.apache.jackrabbit.oak.plugins.version.VersionEditor.propertyChanged(VersionEditor.java:152)
> at
> org.apache.jackrabbit.oak.spi.commit.VisibleEditor.propertyChanged(VisibleEditor.java:73)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.propertyChanged(EditorDiff.java:92)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeState.compareProperties(SegmentNodeState.java:664)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:558)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
> at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:495)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:651)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
> at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:422)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:651)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
> at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:422)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:651)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
> at org.apache.jackrabbit.oak.segment.MapRecord.compare(MapRecord.java:422)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:651)
> at
> org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:147)
> at
>
[jira] [Commented] (OAK-10812) DocumentNodeStore#diffManyChildren(...) may produce incorrect results in readonly mode
[ https://issues.apache.org/jira/browse/OAK-10812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17855965#comment-17855965 ] Stefan Egli commented on OAK-10812: --- [~baedke], you're right, that's a bit odd though. pmin and pmax behave totally different - pmax doesn't remove any clusterIds, while pmin does .. then pmin indeed doesn't look like what's needed. I'm wondering though if a "pmin variant that doesn't remove clusterIds" wouldn't be needed - but this might just be based on the assumption that this thing must be symmetric ... I'd have to come up with a test though to dissect how pmin removing a clusterId is different from a "just fromRev" ... > DocumentNodeStore#diffManyChildren(...) may produce incorrect results in > readonly mode > -- > > Key: OAK-10812 > URL: https://issues.apache.org/jira/browse/OAK-10812 > Project: Jackrabbit Oak > Issue Type: Bug > Components: documentmk >Reporter: Manfred Baedke >Assignee: Manfred Baedke >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10890) Logging for constraint violations (UUID already exists)
[ https://issues.apache.org/jira/browse/OAK-10890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10890: -- Fix Version/s: 1.66.0 > Logging for constraint violations (UUID already exists) > --- > > Key: OAK-10890 > URL: https://issues.apache.org/jira/browse/OAK-10890 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Ionut Pirlogea >Priority: Trivial > Fix For: 1.66.0 > > > In cases where we notice a "uuid already exists" constraint violation - > something that eg can happen when reimporting the same content which was > previously semi-imported - we should increment a metric and ensure we log the > required information (unless the latter is already done through the current > exception - but even then we might want to add perhaps an additional, > explicit log about just the encounter, independent-of/in-addition-to the > exception). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Deleted] (OAK-10902) Logging for constraint violations (UUID already exists)
[ https://issues.apache.org/jira/browse/OAK-10902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli deleted OAK-10902: -- > Logging for constraint violations (UUID already exists) > --- > > Key: OAK-10902 > URL: https://issues.apache.org/jira/browse/OAK-10902 > Project: Jackrabbit Oak > Issue Type: Task >Reporter: Stefan Egli >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10890) Logging for constraint violations (UUID already exists)
[ https://issues.apache.org/jira/browse/OAK-10890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10890: -- Description: In cases where we notice a "uuid already exists" constraint violation - something that eg can happen when reimporting the same content which was previously semi-imported - we should increment a metric and ensure we log the required information (unless the latter is already done through the current exception - but even then we might want to add perhaps an additional, explicit log about just the encounter, independent-of/in-addition-to the exception). (was: In cases where we notice a "uuid already exists" constraint violation - something that eg can happen in Content Backflow reimporting the same content which was previously semi-imported - we should increment a metric and ensure we log the required information (unless the latter is already done through the current exception - but even then we might want to add perhaps an additional, explicit log about just the encounter, independent-of/in-addition-to the exception).) > Logging for constraint violations (UUID already exists) > --- > > Key: OAK-10890 > URL: https://issues.apache.org/jira/browse/OAK-10890 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Ionut Pirlogea >Priority: Trivial > > In cases where we notice a "uuid already exists" constraint violation - > something that eg can happen when reimporting the same content which was > previously semi-imported - we should increment a metric and ensure we log the > required information (unless the latter is already done through the current > exception - but even then we might want to add perhaps an additional, > explicit log about just the encounter, independent-of/in-addition-to the > exception). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OAK-10902) Logging for constraint violations (UUID already exists)
Stefan Egli created OAK-10902: - Summary: Logging for constraint violations (UUID already exists) Key: OAK-10902 URL: https://issues.apache.org/jira/browse/OAK-10902 Project: Jackrabbit Oak Issue Type: Task Components: documentmk Reporter: Stefan Egli -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] (OAK-10890) Logging for constraint violations (UUID already exists)
[ https://issues.apache.org/jira/browse/OAK-10890 ] Stefan Egli deleted comment on OAK-10890: --- was (Author: JIRAUSER305793): ERROR message encounter in splunk: msg: [qtp1617000337-4715] org.apache.jackrabbit.vault.fs.io.AutoSave Error during auto save, retrying after refresh: javax.jcr.nodetype.ConstraintViolationException: OakConstraint0030: Uniqueness constraint violated property [jcr:uuid] having value 97a322dc-58c6-4ffb-9338-3f65fafc25c5: /content/heliux/beauty/skp/language-masters/pl/colour/blondme-colour/lifting-and-blending/lift-blend/jcr:content, /content/heliux/beauty/skp/language-masters/pl/colour/blondme-colour1/lifting-and-blending/lift-blend/jcr:content. Caused by org.apache.jackrabbit.oak.api.CommitFailedException: OakConstraint0030: Uniqueness constraint violated property [jcr:uuid] having value 97a322dc-58c6-4ffb-9338-3f65fafc25c5: /content/heliux/beauty/skp/language-masters/pl/colour/blondme-colour/lifting-and-blending/lift-blend/jcr:content, /content/heliux/beauty/skp/language-masters/pl/colour/blondme-colour1/lifting-and-blending/lift-blend/jcr:content > Logging for constraint violations (UUID already exists) > --- > > Key: OAK-10890 > URL: https://issues.apache.org/jira/browse/OAK-10890 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Ionut Pirlogea >Priority: Trivial > > In cases where we notice a "uuid already exists" constraint violation - > something that eg can happen when reimporting the same content which was > previously semi-imported - we should increment a metric and ensure we log the > required information (unless the latter is already done through the current > exception - but even then we might want to add perhaps an additional, > explicit log about just the encounter, independent-of/in-addition-to the > exception). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10845) store-document: occasional test failure: BranchCommitGCTest.unmergedMergedRemoveChild
[ https://issues.apache.org/jira/browse/OAK-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17855604#comment-17855604 ] Stefan Egli commented on OAK-10845: --- Another flaky test combination removal PR : https://github.com/apache/jackrabbit-oak/pull/1536 > store-document: occasional test failure: > BranchCommitGCTest.unmergedMergedRemoveChild > - > > Key: OAK-10845 > URL: https://issues.apache.org/jira/browse/OAK-10845 > Project: Jackrabbit Oak > Issue Type: Bug > Components: documentmk >Reporter: Julian Reschke >Priority: Minor > > [ERROR] Failures: > [ERROR] BranchCommitGCTest.unmergedMergedRemoveChild:623 > ORPHANS_EMPTYPROPS_BETWEEN_CHECKPOINTS_WITH_UNMERGED_BC/internalProps > expected:<2> but was:<1> -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10880) Logging for hitting an orphaned UUIDs and reference indexes.
[ https://issues.apache.org/jira/browse/OAK-10880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17854696#comment-17854696 ] Stefan Egli commented on OAK-10880: --- Upon revisiting this, I believe introducing a logging as is described in this ticket is actually not possible. The parameter passed to session.getNodeByUUID/.getNodeByIdentifier doesn't come with any guarantee as to where it comes from. The caller could have made it up - it could be a dangling id it keeps in a non-reference property - it could be a proper one as well. Everything is possible there. The fact that the node cannot be found doesn't necessarily imply that /oak:index/uuid contains a dangling pointer (for which it was the intention to log). I don't think we can implement this. > Logging for hitting an orphaned UUIDs and reference indexes. > > > Key: OAK-10880 > URL: https://issues.apache.org/jira/browse/OAK-10880 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Ionut Pirlogea >Priority: Minor > > Lookups of a node by uuid (getNodeByUuid) can hit a uuid index entry that > points to a no-longer-existing node - i.e. it can point to an orphaned node, > and thereby the uuid index entry is also "kind of orphaned" (but it can > actually be reached via the jcr tree). Upon such an encounter, we should > increment a metric and log the encounter with relevant information (eg just > the uuid), if that's not already done. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10881) Expand oak/docs/participating with some more guidelines
[ https://issues.apache.org/jira/browse/OAK-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10881: -- Description: Proposal: Tickets/PRs/Workflow: - Always reference an Oak ticket for each commit/PR (this should include the jira id in the correct format, eg OAK-10881 instead of Oak 10881) - minimize PRs; do not modify whitespace/coding style except where needed - structure tickets/PRs so that things that can be separated are (that can be useful for backports) - have test cases (when there's no immediate fix, create a ticket and a PR just for the test and mark it "ignored", pointing to the actual issue) - trunk: when done with a ticket, set it to "resolved" and set "Fix Version" to the next unreleased version - maintenance branch: re-use the existing Jira ticket and just add to "Fix Version" (unless the backport is complex) - avoid committing unfinished stuff to trunk; in particular when a release is approaching - add affects-version and fix-version as and when applicable - PRs that contain multiple commits in general should be "squashed and merged" Coding Style: - no wildcard imports - in general be consistent with the style of the code being modified - avoid TABs and trailing whitespace was: Proposal: Tickets/PRs/Workflow: - Always reference an Oak ticket for each commit/PR - minimize PRs; do not modify whitespace/coding style except where needed - structure tickets/PRs so that things that can be separated are (that can be useful for backports) - have test cases (when there's no immediate fix, create a ticket and a PR just for the test and mark it "ignored", pointing to the actual issue) - trunk: when done with a ticket, set it to "resolved" and set "Fix Version" to the next unreleased version - maintenance branch: re-use the existing Jira ticket and just add to "Fix Version" (unless the backport is complex) - avoid committing unfinished stuff to trunk; in particular when a release is approaching - add affects-version and fix-version as and when applicable - PRs that contain multiple commits in general should be "squashed and merged" Coding Style: - no wildcard imports - in general be consistent with the style of the code being modified - avoid TABs and trailing whitespace > Expand oak/docs/participating with some more guidelines > --- > > Key: OAK-10881 > URL: https://issues.apache.org/jira/browse/OAK-10881 > Project: Jackrabbit Oak > Issue Type: Documentation > Components: doc >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > > Proposal: > Tickets/PRs/Workflow: > - Always reference an Oak ticket for each commit/PR (this should include the > jira id in the correct format, eg OAK-10881 instead of Oak 10881) > - minimize PRs; do not modify whitespace/coding style except where needed > - structure tickets/PRs so that things that can be separated are (that can be > useful for backports) > - have test cases (when there's no immediate fix, create a ticket and a PR > just for the test and mark it "ignored", pointing to the actual issue) > - trunk: when done with a ticket, set it to "resolved" and set "Fix Version" > to the next unreleased version > - maintenance branch: re-use the existing Jira ticket and just add to "Fix > Version" (unless the backport is complex) > - avoid committing unfinished stuff to trunk; in particular when a release is > approaching > - add affects-version and fix-version as and when applicable > - PRs that contain multiple commits in general should be "squashed and merged" > Coding Style: > - no wildcard imports > - in general be consistent with the style of the code being modified > - avoid TABs and trailing whitespace -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10880) Logging for hitting an orphaned UUIDs and reference indexes.
[ https://issues.apache.org/jira/browse/OAK-10880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17854697#comment-17854697 ] Stefan Egli commented on OAK-10880: --- FTR : PR covering a suggestion for this is at https://github.com/apache/jackrabbit-oak/pull/1520 > Logging for hitting an orphaned UUIDs and reference indexes. > > > Key: OAK-10880 > URL: https://issues.apache.org/jira/browse/OAK-10880 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Ionut Pirlogea >Priority: Minor > > Lookups of a node by uuid (getNodeByUuid) can hit a uuid index entry that > points to a no-longer-existing node - i.e. it can point to an orphaned node, > and thereby the uuid index entry is also "kind of orphaned" (but it can > actually be reached via the jcr tree). Upon such an encounter, we should > increment a metric and log the encounter with relevant information (eg just > the uuid), if that's not already done. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (OAK-10880) Logging for hitting an orphaned UUIDs and reference indexes.
[ https://issues.apache.org/jira/browse/OAK-10880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli resolved OAK-10880. --- Resolution: Invalid ok, closing as invalid. > Logging for hitting an orphaned UUIDs and reference indexes. > > > Key: OAK-10880 > URL: https://issues.apache.org/jira/browse/OAK-10880 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Ionut Pirlogea >Priority: Minor > > Lookups of a node by uuid (getNodeByUuid) can hit a uuid index entry that > points to a no-longer-existing node - i.e. it can point to an orphaned node, > and thereby the uuid index entry is also "kind of orphaned" (but it can > actually be reached via the jcr tree). Upon such an encounter, we should > increment a metric and log the encounter with relevant information (eg just > the uuid), if that's not already done. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10881) Expand oak/docs/participating with some more guidelines
[ https://issues.apache.org/jira/browse/OAK-10881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10881: -- Description: Proposal: Tickets/PRs/Workflow: - Always reference an Oak ticket for each commit/PR - minimize PRs; do not modify whitespace/coding style except where needed - structure tickets/PRs so that things that can be separated are (that can be useful for backports) - have test cases (when there's no immediate fix, create a ticket and a PR just for the test and mark it "ignored", pointing to the actual issue) - trunk: when done with a ticket, set it to "resolved" and set "fixVersion" to the next unreleased version - maintenance branch: re-use the existing Jira ticket and just add to "fixVersion" (unless the backport is complex) - avoid committing unfinished stuff to trunk; in particular when a release is approaching - add affects-version and fix-version as and when applicable Coding Style: - no wildcard imports - in general be consistent with the style of the code being modified - avoid TABs and trailing whitespace was: Proposal: Tickets/PRs/Workflow: - Always reference an Oak ticket for each commit/PR - minimize PRs; do not modify whitespace/coding style except where needed - structure tickets/PRs so that things that can be separated are (that can be useful for backports) - have test cases (when there's no immediate fix, create a ticket and a PR just for the test and mark it "ignored", pointing to the actual issue) - trunk: when done with a ticket, set it to "resolved" and set "fixVersion" to the next unreleased version - maintenance branch: re-use the existing Jira ticket and just add to "fixVersion" (unless the backport is complex) - avoid committing unfinished stuff to trunk; in particular when a release is approaching Coding Style: - no wildcard imports - in general be consistent with the style of the code being modified - avoid TABs and trailing whitespace > Expand oak/docs/participating with some more guidelines > --- > > Key: OAK-10881 > URL: https://issues.apache.org/jira/browse/OAK-10881 > Project: Jackrabbit Oak > Issue Type: Documentation > Components: doc >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > > Proposal: > Tickets/PRs/Workflow: > - Always reference an Oak ticket for each commit/PR > - minimize PRs; do not modify whitespace/coding style except where needed > - structure tickets/PRs so that things that can be separated are (that can be > useful for backports) > - have test cases (when there's no immediate fix, create a ticket and a PR > just for the test and mark it "ignored", pointing to the actual issue) > - trunk: when done with a ticket, set it to "resolved" and set "fixVersion" > to the next unreleased version > - maintenance branch: re-use the existing Jira ticket and just add to > "fixVersion" (unless the backport is complex) > - avoid committing unfinished stuff to trunk; in particular when a release is > approaching > - add affects-version and fix-version as and when applicable > Coding Style: > - no wildcard imports > - in general be consistent with the style of the code being modified > - avoid TABs and trailing whitespace -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (OAK-10803) Compress in-memory property values
[ https://issues.apache.org/jira/browse/OAK-10803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17854399#comment-17854399 ] Stefan Egli edited comment on OAK-10803 at 6/12/24 11:14 AM: - +1 to -both- all 1) I think in general we are more and more introducing new configs in different places but have different support on how that config can be modified. It might be useful to simplify and perhaps consolidate the different options.. 2) +1. There was also the option to not even doing compression if it wasn't worth it - but the current code doesn't go that far at this point. Perhaps requiring a minimal memory saving config option would indeed be useful. 3) +1 for a metric was (Author: egli): +1 to both 1) I think in general we are more and more introducing new configs in different places but have different support on how that config can be modified. It might be useful to simplify and perhaps consolidate the different options.. 2) +1. There was also the option to not even doing compression if it wasn't worth it - but the current code doesn't go that far at this point. Perhaps requiring a minimal memory saving config option would indeed be useful. 3) +1 for a metric > Compress in-memory property values > -- > > Key: OAK-10803 > URL: https://issues.apache.org/jira/browse/OAK-10803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Daniel Iancu >Priority: Major > Fix For: 1.66.0 > > > Some properties in the repository can be quite large and consume a > considerable amount of memory. E.g. large multi-valued properties, blob IDs > that contain an inlined binary. > With a certain size of the value, it may be beneficial to compress it when > loaded into memory/cache and uncompress it only on access. This shouldn't be > too difficult and can be implemented in a backward compatible way, because it > only affects the in-memory representation and not how data is stored. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10803) Compress in-memory property values
[ https://issues.apache.org/jira/browse/OAK-10803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17854399#comment-17854399 ] Stefan Egli commented on OAK-10803: --- +1 to both 1) I think in general we are more and more introducing new configs in different places but have different support on how that config can be modified. It might be useful to simplify and perhaps consolidate the different options.. 2) +1. There was also the option to not even doing compression if it wasn't worth it - but the current code doesn't go that far at this point. Perhaps requiring a minimal memory saving config option would indeed be useful. 3) +1 for a metric > Compress in-memory property values > -- > > Key: OAK-10803 > URL: https://issues.apache.org/jira/browse/OAK-10803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Daniel Iancu >Priority: Major > Fix For: 1.66.0 > > > Some properties in the repository can be quite large and consume a > considerable amount of memory. E.g. large multi-valued properties, blob IDs > that contain an inlined binary. > With a certain size of the value, it may be beneficial to compress it when > loaded into memory/cache and uncompress it only on access. This shouldn't be > too difficult and can be implemented in a backward compatible way, because it > only affects the in-memory representation and not how data is stored. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (OAK-10803) Compress in-memory property values
[ https://issues.apache.org/jira/browse/OAK-10803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli resolved OAK-10803. --- Fix Version/s: 1.68.0 Resolution: Done * merged the PR : https://github.com/apache/jackrabbit-oak/pull/1473 * thx for the contributions Ionut! * marking ticket done > Compress in-memory property values > -- > > Key: OAK-10803 > URL: https://issues.apache.org/jira/browse/OAK-10803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Daniel Iancu >Priority: Major > Fix For: 1.68.0 > > > Some properties in the repository can be quite large and consume a > considerable amount of memory. E.g. large multi-valued properties, blob IDs > that contain an inlined binary. > With a certain size of the value, it may be beneficial to compress it when > loaded into memory/cache and uncompress it only on access. This shouldn't be > too difficult and can be implemented in a backward compatible way, because it > only affects the in-memory representation and not how data is stored. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10803) Compress in-memory property values
[ https://issues.apache.org/jira/browse/OAK-10803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10803: -- Component/s: documentmk (was: core) > Compress in-memory property values > -- > > Key: OAK-10803 > URL: https://issues.apache.org/jira/browse/OAK-10803 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Reporter: Daniel Iancu >Priority: Major > > Some properties in the repository can be quite large and consume a > considerable amount of memory. E.g. large multi-valued properties, blob IDs > that contain an inlined binary. > With a certain size of the value, it may be beneficial to compress it when > loaded into memory/cache and uncompress it only on access. This shouldn't be > too difficult and can be implemented in a backward compatible way, because it > only affects the in-memory representation and not how data is stored. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (OAK-10870) New oak-mongo.js methods for sizes of properties, counts by clusterId and unmerged branch commits
[ https://issues.apache.org/jira/browse/OAK-10870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli resolved OAK-10870. --- Fix Version/s: 1.68.0 Resolution: Done PR merged to trunk, marking ticket done. > New oak-mongo.js methods for sizes of properties, counts by clusterId and > unmerged branch commits > - > > Key: OAK-10870 > URL: https://issues.apache.org/jira/browse/OAK-10870 > Project: Jackrabbit Oak > Issue Type: Task > Components: run >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > Fix For: 1.68.0 > > > A few new utility methods in oak-run's oak-mongo.js for > * property revision size stats > * property revision counts by clusterId > * all branch commit values in a document > * unmerged branch commits by property > And a method for counting direct children (vs all descendants) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10875) unmerged bc on root can make it appear merged
[ https://issues.apache.org/jira/browse/OAK-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853964#comment-17853964 ] Stefan Egli commented on OAK-10875: --- Created test case to reproduce this at PR https://github.com/apache/jackrabbit-oak/pull/1517 > unmerged bc on root can make it appear merged > - > > Key: OAK-10875 > URL: https://issues.apache.org/jira/browse/OAK-10875 > Project: Jackrabbit Oak > Issue Type: Bug > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > If an oak instance crashes during a branch commit, which in turns leaves the > bc unmerged - and that branch contained changes on root - it can later result > in that unmerged bc to be considered as merged. If that values is then cached > before other changes of the branch are read, those other changes are then > considered merged even though they never have. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OAK-10875) unmerged bc on root can make it appear merged
Stefan Egli created OAK-10875: - Summary: unmerged bc on root can make it appear merged Key: OAK-10875 URL: https://issues.apache.org/jira/browse/OAK-10875 Project: Jackrabbit Oak Issue Type: Bug Components: documentmk Reporter: Stefan Egli If an oak instance crashes during a branch commit, which in turns leaves the bc unmerged - and that branch contained changes on root - it can later result in that unmerged bc to be considered as merged. If that values is then cached before other changes of the branch are read, those other changes are then considered merged even though they never have. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10870) New oak-mongo.js methods for sizes of properties, counts by clusterId and unmerged branch commits
[ https://issues.apache.org/jira/browse/OAK-10870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853685#comment-17853685 ] Stefan Egli commented on OAK-10870: --- PR created at https://github.com/apache/jackrabbit-oak/compare/OAK-10870?expand=1 > New oak-mongo.js methods for sizes of properties, counts by clusterId and > unmerged branch commits > - > > Key: OAK-10870 > URL: https://issues.apache.org/jira/browse/OAK-10870 > Project: Jackrabbit Oak > Issue Type: Task > Components: run >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > > A few new utility methods in oak-run's oak-mongo.js for > * property revision size stats > * property revision counts by clusterId > * all branch commit values in a document > * unmerged branch commits by property > And a method for counting direct children (vs all descendants) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (OAK-10870) New oak-mongo.js methods for sizes of properties, counts by clusterId and unmerged branch commits
[ https://issues.apache.org/jira/browse/OAK-10870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853685#comment-17853685 ] Stefan Egli edited comment on OAK-10870 at 6/10/24 2:05 PM: PR created at https://github.com/apache/jackrabbit-oak/pull/1514 was (Author: egli): PR created at https://github.com/apache/jackrabbit-oak/compare/OAK-10870?expand=1 > New oak-mongo.js methods for sizes of properties, counts by clusterId and > unmerged branch commits > - > > Key: OAK-10870 > URL: https://issues.apache.org/jira/browse/OAK-10870 > Project: Jackrabbit Oak > Issue Type: Task > Components: run >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > > A few new utility methods in oak-run's oak-mongo.js for > * property revision size stats > * property revision counts by clusterId > * all branch commit values in a document > * unmerged branch commits by property > And a method for counting direct children (vs all descendants) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OAK-10870) New oak-mongo.js methods for sizes of properties, counts by clusterId and unmerged branch commits
Stefan Egli created OAK-10870: - Summary: New oak-mongo.js methods for sizes of properties, counts by clusterId and unmerged branch commits Key: OAK-10870 URL: https://issues.apache.org/jira/browse/OAK-10870 Project: Jackrabbit Oak Issue Type: Task Components: run Reporter: Stefan Egli Assignee: Stefan Egli A few new utility methods in oak-run's oak-mongo.js for * property revision size stats * property revision counts by clusterId * all branch commit values in a document * unmerged branch commits by property And a method for counting direct children (vs all descendants) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (OAK-10869) testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode test failure
[ https://issues.apache.org/jira/browse/OAK-10869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli reassigned OAK-10869: - Assignee: Stefan Egli > testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode test failure > - > > Key: OAK-10869 > URL: https://issues.apache.org/jira/browse/OAK-10869 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Assignee: Stefan Egli >Priority: Major > > https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/org.apache.jackrabbit$oak-store-document/1524/testReport/junit/org.apache.jackrabbit.oak.plugins.document/VersionGarbageCollectorIT/testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode_4__MongoFixture__MongoDB_with_GAP_ORPHANS_EMPTYPROPS_/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10869) testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode test failure
[ https://issues.apache.org/jira/browse/OAK-10869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10869: -- Epic Link: OAK-10739 > testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode test failure > - > > Key: OAK-10869 > URL: https://issues.apache.org/jira/browse/OAK-10869 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/org.apache.jackrabbit$oak-store-document/1524/testReport/junit/org.apache.jackrabbit.oak.plugins.document/VersionGarbageCollectorIT/testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode_4__MongoFixture__MongoDB_with_GAP_ORPHANS_EMPTYPROPS_/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OAK-10869) testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode test failure
Stefan Egli created OAK-10869: - Summary: testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode test failure Key: OAK-10869 URL: https://issues.apache.org/jira/browse/OAK-10869 Project: Jackrabbit Oak Issue Type: Task Components: documentmk Reporter: Stefan Egli https://ci-builds.apache.org/job/Jackrabbit/job/jackrabbit-oak-trunk/org.apache.jackrabbit$oak-store-document/1524/testReport/junit/org.apache.jackrabbit.oak.plugins.document/VersionGarbageCollectorIT/testDeletedPropsAndUnmergedBCWithCollisionWithDryRunMode_4__MongoFixture__MongoDB_with_GAP_ORPHANS_EMPTYPROPS_/ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10812) DocumentNodeStore#diffManyChildren(...) may produce incorrect results in readonly mode
[ https://issues.apache.org/jira/browse/OAK-10812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17852853#comment-17852853 ] Stefan Egli commented on OAK-10812: --- (maybe the two methods could be combined to avoid code duplication.. - I didn't get to do that part though) > DocumentNodeStore#diffManyChildren(...) may produce incorrect results in > readonly mode > -- > > Key: OAK-10812 > URL: https://issues.apache.org/jira/browse/OAK-10812 > Project: Jackrabbit Oak > Issue Type: Bug > Components: documentmk >Reporter: Manfred Baedke >Assignee: Manfred Baedke >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10812) DocumentNodeStore#diffManyChildren(...) may produce incorrect results in readonly mode
[
https://issues.apache.org/jira/browse/OAK-10812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17852852#comment-17852852
]
Stefan Egli commented on OAK-10812:
---
For reference, here's the read-write test method I mentioned:
{code:java}
@Test
public void diffManyChildrenReadWriteMode() throws Exception {
DocumentMK mk1 = createMK(1, 0);
DocumentMK mk2 = createMK(2, 0);
NodeBuilder builder = mk1.getNodeStore().getRoot().builder();
builder.child("foo1").child("bar1");
merge(mk1.getNodeStore(), builder);
mk1.runBackgroundOperations();
mk2.runBackgroundOperations();
RevisionVector fromRev = mk1.getNodeStore().getRoot().getLastRevision();
Thread.sleep(1000);
builder = mk1.getNodeStore().getRoot().builder();
builder.getChildNode("foo1").getChildNode("bar1").setProperty("test",
"test");
merge(mk1.getNodeStore(), builder);
disposeMK(mk1);
Thread.sleep(1000);
mk1 = createMK(1, 0);
DocumentMK mk3rw = createMK(3, 0, false);
DocumentNodeStore ns3rw = mk3rw.getNodeStore();
RevisionVector toRev = ns3rw.getRoot().getLastRevision();
Thread.sleep(5000);
JsopWriter w1 = new JsopStream();
ns3rw.diffManyChildren(w1, ns3rw.getRoot().getPath(), fromRev, toRev);
JsopWriter w2 = new JsopStream();
mk1.getNodeStore().diffManyChildren(w2,
mk1.nodeStore.getRoot().getPath(), fromRev, toRev);
assertEquals(w1.toString(), w2.toString());
}
{code}
> DocumentNodeStore#diffManyChildren(...) may produce incorrect results in
> readonly mode
> --
>
> Key: OAK-10812
> URL: https://issues.apache.org/jira/browse/OAK-10812
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: documentmk
>Reporter: Manfred Baedke
>Assignee: Manfred Baedke
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10812) DocumentNodeStore#diffManyChildren(...) may produce incorrect results in readonly mode
[
https://issues.apache.org/jira/browse/OAK-10812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17852849#comment-17852849
]
Stefan Egli commented on OAK-10812:
---
{quote}I think that we have to use the parallel minimum instead of the parallel
maximum in this case. {quote}
Right, I think the bug lies in the fact that {{pmax}} was used for {{fromRev}}
*and* {{toRev}} - while it seems like the correct way would have been to use
{{pmin}} for {{fromRev}} and {{pmax}} for {{toRev}}.
Except, in the current state of
[PR#1465|https://github.com/apache/jackrabbit-oak/pull/1465] that's not what
the PR now contains. Instead of switching pmax to pmin it now does a read-only
conditioned ignore of minRevision. I think that following the above mentioned
conclusion, it should actually be a simple flip of
{noformat}
-fromRev = fromRev.pmax(minRevisions);
+fromRev = fromRev.pmin(minRevisions);
{noformat}
Btw, what might also be useful is to have that new diffManyChildrenReadOnlyMode
test method in two variants (and then maybe call it different) : one for
read-only and one for read-write. I understand the read-write one wouldn't be
able to reuse clusterId 1 - as is currently coded in
diffManyChildrenReadOnlyMode. So it would probably use clusterId 3. I have
tested a read-write variant of that method and it indeed fails with the current
code (that applies only to read-only mode - but in fact, the bug is independent
of read-only mode).
> DocumentNodeStore#diffManyChildren(...) may produce incorrect results in
> readonly mode
> --
>
> Key: OAK-10812
> URL: https://issues.apache.org/jira/browse/OAK-10812
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: documentmk
>Reporter: Manfred Baedke
>Assignee: Manfred Baedke
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (OAK-10837) Add documentation for UT/IT fixtures
[ https://issues.apache.org/jira/browse/OAK-10837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10837: -- Component/s: doc (was: continuous integration) > Add documentation for UT/IT fixtures > > > Key: OAK-10837 > URL: https://issues.apache.org/jira/browse/OAK-10837 > Project: Jackrabbit Oak > Issue Type: Task > Components: doc >Reporter: Adelina Marian >Priority: Minor > > Improve the current documentation to bring clarity about how to run unit and > integration tests locally with different fixtures. > Add an overview of these configurations on our CI pipelines on the Jenkins > instances. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (OAK-10837) Add documentation for UT/IT fixtures
[ https://issues.apache.org/jira/browse/OAK-10837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli resolved OAK-10837. --- Fix Version/s: 1.66.0 Resolution: Fixed Thx for the PR, merged now, marking ticket as done. > Add documentation for UT/IT fixtures > > > Key: OAK-10837 > URL: https://issues.apache.org/jira/browse/OAK-10837 > Project: Jackrabbit Oak > Issue Type: Task > Components: doc >Reporter: Adelina Marian >Priority: Minor > Fix For: 1.66.0 > > > Improve the current documentation to bring clarity about how to run unit and > integration tests locally with different fixtures. > Add an overview of these configurations on our CI pipelines on the Jenkins > instances. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10852) oak-store-document: disable unreliable tests in BranchCommitGCTest
[ https://issues.apache.org/jira/browse/OAK-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851651#comment-17851651 ] Stefan Egli commented on OAK-10852: --- refinement to 46476bcbfc : https://github.com/apache/jackrabbit-oak/pull/1504 > oak-store-document: disable unreliable tests in BranchCommitGCTest > -- > > Key: OAK-10852 > URL: https://issues.apache.org/jira/browse/OAK-10852 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: documentmk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Fix For: 1.66.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (OAK-10852) oak-store-document: disable unreliable tests in BranchCommitGCTest
[ https://issues.apache.org/jira/browse/OAK-10852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851570#comment-17851570 ] Stefan Egli edited comment on OAK-10852 at 6/3/24 2:05 PM: --- trunk: [46476bcbfc|https://github.com/apache/jackrabbit-oak/commit/46476bcbfca961834780c162a4266f0b305a06d9] was (Author: reschke): trunk: [46476bcbfc|https://github.com/apache/jackrabbit-oak-/commit/46476bcbfca961834780c162a4266f0b305a06d9] > oak-store-document: disable unreliable tests in BranchCommitGCTest > -- > > Key: OAK-10852 > URL: https://issues.apache.org/jira/browse/OAK-10852 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: documentmk >Reporter: Julian Reschke >Assignee: Julian Reschke >Priority: Minor > Fix For: 1.66.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10853) VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithCollision failure
[
https://issues.apache.org/jira/browse/OAK-10853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851615#comment-17851615
]
Stefan Egli commented on OAK-10853:
---
PR for temporary disable -> https://github.com/apache/jackrabbit-oak/pull/1502
> VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithCollision failure
>
>
> Key: OAK-10853
> URL: https://issues.apache.org/jira/browse/OAK-10853
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: documentmk, test
>Reporter: Julian Reschke
>Priority: Major
>
> {noformat}
> [ERROR] Tests run: 671, Failures: 1, Errors: 0, Skipped: 89, Time elapsed:
> 1,404.262 s <<< FAILURE! - in org.apache.jackrabbit.oak.plugins.d
> ocument.VersionGarbageCollectorIT
> [ERROR] testDeletedPropsAndUnmergedBCWithCollision[7: MongoFixture: MongoDB
> with ORPHANS_EMPTYPROPS_KEEP_ONE_ALL_PROPS](org.apache.jackrabbi
> t.oak.plugins.document.VersionGarbageCollectorIT) Time elapsed: 0.432 s <<<
> FAILURE!
> java.lang.AssertionError: ORPHANS_EMPTYPROPS_KEEP_ONE_ALL_PROPS/internalProps
> expected:<2> but was:<1>
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotEquals(Assert.java:835)
> at org.junit.Assert.assertEquals(Assert.java:647)
> at
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorIT.assertStatsCountsEqual(VersionGarbageCollectorIT.java:1229)
> at
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorIT.testDeletedPropsAndUnmergedBCWithCollision(VersionGarbageCol
> lectorIT.java:1678)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
>
[jira] [Commented] (OAK-10845) store-document: occasional test failure: BranchCommitGCTest.unmergedMergedRemoveChild
[ https://issues.apache.org/jira/browse/OAK-10845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851556#comment-17851556 ] Stefan Egli commented on OAK-10845: --- +1 for disabling for now. We have loads of test combinations such that disabling individual flakyness doesn't reduce test coverage noticeably in my view. But having the flaky tests adds cost - so when we see them I'd argue we should disable them (and work on a fix asynchronously). > store-document: occasional test failure: > BranchCommitGCTest.unmergedMergedRemoveChild > - > > Key: OAK-10845 > URL: https://issues.apache.org/jira/browse/OAK-10845 > Project: Jackrabbit Oak > Issue Type: Bug > Components: documentmk >Reporter: Julian Reschke >Priority: Minor > > [ERROR] Failures: > [ERROR] BranchCommitGCTest.unmergedMergedRemoveChild:623 > ORPHANS_EMPTYPROPS_BETWEEN_CHECKPOINTS_WITH_UNMERGED_BC/internalProps > expected:<2> but was:<1> -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10846) testPartialMergeRootCleanup leaks thread
[ https://issues.apache.org/jira/browse/OAK-10846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850774#comment-17850774 ] Stefan Egli commented on OAK-10846: --- PR created to temporarily remove it until we have it reviewed and fixed -> https://github.com/apache/jackrabbit-oak/pull/1499 > testPartialMergeRootCleanup leaks thread > > > Key: OAK-10846 > URL: https://issues.apache.org/jira/browse/OAK-10846 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > VersionGarbageCollectorIT.testPartialMergeRootCleanup creates a second store > twice - the first one it lets fail eternally, but that seems to leak a thread. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OAK-10846) testPartialMergeRootCleanup leaks thread
Stefan Egli created OAK-10846: - Summary: testPartialMergeRootCleanup leaks thread Key: OAK-10846 URL: https://issues.apache.org/jira/browse/OAK-10846 Project: Jackrabbit Oak Issue Type: Task Components: documentmk Reporter: Stefan Egli VersionGarbageCollectorIT.testPartialMergeRootCleanup creates a second store twice - the first one it lets fail eternally, but that seems to leak a thread. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10844) speed up fullgc tests
[ https://issues.apache.org/jira/browse/OAK-10844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850759#comment-17850759 ] Stefan Egli commented on OAK-10844: --- FTR : that flaky test-combination testBundledPropUnmergedBCGC/ORPHANS_EMPTYPROPS_KEEP_ONE_ALL_PROPS was disabled [over in OAK-10843|https://issues.apache.org/jira/browse/OAK-10843?focusedCommentId=17850687=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17850687] > speed up fullgc tests > - > > Key: OAK-10844 > URL: https://issues.apache.org/jira/browse/OAK-10844 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > The new fullgc tests are unacceptably slow on some environments (in the range > of 2h). It might have to do with mongo in docker - but it's also multiplied > with many permutations of fullgcmode and fixtures that are run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10844) speed up fullgc tests
[ https://issues.apache.org/jira/browse/OAK-10844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850740#comment-17850740 ] Stefan Egli commented on OAK-10844: --- Another PR to ignore a slow test : https://github.com/apache/jackrabbit-oak/pull/1498 > speed up fullgc tests > - > > Key: OAK-10844 > URL: https://issues.apache.org/jira/browse/OAK-10844 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > The new fullgc tests are unacceptably slow on some environments (in the range > of 2h). It might have to do with mongo in docker - but it's also multiplied > with many permutations of fullgcmode and fixtures that are run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10843) Flaky fullgc tests
[ https://issues.apache.org/jira/browse/OAK-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850688#comment-17850688 ] Stefan Egli commented on OAK-10843: --- PR for that flaky run -> https://github.com/apache/jackrabbit-oak/pull/1497 > Flaky fullgc tests > -- > > Key: OAK-10843 > URL: https://issues.apache.org/jira/browse/OAK-10843 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > As noted in OAK-10739 there is potential flakyness in tests with fullgc modes > "BETWEEN_CHECKPOINTS" and others. This ticket is to look into fixing it. > First measure could be to disable them until we have a fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10843) Flaky fullgc tests
[
https://issues.apache.org/jira/browse/OAK-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850687#comment-17850687
]
Stefan Egli commented on OAK-10843:
---
[~reschke], I'm going to exclude the following flaky run :
{noformat}
[ERROR] testBundledPropUnmergedBCGC[7: MongoFixture: MongoDB with
ORPHANS_EMPTYPROPS_KEEP_ONE_ALL_PROPS](org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorIT)
Time elapsed: 0.578 s <<< FAILURE!
java.lang.AssertionError: ORPHANS_EMPTYPROPS_KEEP_ONE_ALL_PROPS/internalProps
expected:<2> but was:<1>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorIT.assertStatsCountsEqual(VersionGarbageCollectorIT.java:1227)
at
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorIT.testBundledPropUnmergedBCGC(VersionGarbageCollectorIT.java:1943)
{noformat}
> Flaky fullgc tests
> --
>
> Key: OAK-10843
> URL: https://issues.apache.org/jira/browse/OAK-10843
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: documentmk
>Reporter: Stefan Egli
>Priority: Major
>
> As noted in OAK-10739 there is potential flakyness in tests with fullgc modes
> "BETWEEN_CHECKPOINTS" and others. This ticket is to look into fixing it.
> First measure could be to disable them until we have a fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (OAK-10844) speed up fullgc tests
[ https://issues.apache.org/jira/browse/OAK-10844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850616#comment-17850616 ] Stefan Egli commented on OAK-10844: --- Initial PR to (1) reduce nr of test combinations and (2) add logging to help narrowing down where time is spent : https://github.com/apache/jackrabbit-oak/pull/1494 > speed up fullgc tests > - > > Key: OAK-10844 > URL: https://issues.apache.org/jira/browse/OAK-10844 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > The new fullgc tests are unacceptably slow on some environments (in the range > of 2h). It might have to do with mongo in docker - but it's also multiplied > with many permutations of fullgcmode and fixtures that are run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OAK-10844) speed up fullgc tests
Stefan Egli created OAK-10844: - Summary: speed up fullgc tests Key: OAK-10844 URL: https://issues.apache.org/jira/browse/OAK-10844 Project: Jackrabbit Oak Issue Type: Task Components: documentmk Reporter: Stefan Egli The new fullgc tests are unacceptably slow on some environments (in the range of 2h). It might have to do with mongo in docker - but it's also multiplied with many permutations of fullgcmode and fixtures that are run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10844) speed up fullgc tests
[ https://issues.apache.org/jira/browse/OAK-10844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10844: -- Epic Link: OAK-10739 > speed up fullgc tests > - > > Key: OAK-10844 > URL: https://issues.apache.org/jira/browse/OAK-10844 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > The new fullgc tests are unacceptably slow on some environments (in the range > of 2h). It might have to do with mongo in docker - but it's also multiplied > with many permutations of fullgcmode and fixtures that are run. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10843) Flaky fullgc tests
[ https://issues.apache.org/jira/browse/OAK-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10843: -- Epic Link: OAK-10739 > Flaky fullgc tests > -- > > Key: OAK-10843 > URL: https://issues.apache.org/jira/browse/OAK-10843 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > As noted in OAK-10739 there is potential flakyness in tests with fullgc modes > "BETWEEN_CHECKPOINTS" and others. This ticket is to look into fixing it. > First measure could be to disable them until we have a fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10843) Flaky fullgc tests
[ https://issues.apache.org/jira/browse/OAK-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10843: -- Description: As noted in OAK-10739 there is potential flakyness in tests with fullgc modes "BETWEEN_CHECKPOINTS" and others. This ticket is to look into fixing it. First measure could be to disable them until we have a fix. (was: As noted in OAK-10739 there is potential flakyness in tests with fullgc modes "BETWEEN_CHECKPOINTS". This ticket is to look into fixing it. First measure could be to disable them until we have a fix.) > Flaky fullgc tests > -- > > Key: OAK-10843 > URL: https://issues.apache.org/jira/browse/OAK-10843 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > As noted in OAK-10739 there is potential flakyness in tests with fullgc modes > "BETWEEN_CHECKPOINTS" and others. This ticket is to look into fixing it. > First measure could be to disable them until we have a fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (OAK-10843) Flaky fullgc tests
[ https://issues.apache.org/jira/browse/OAK-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Egli updated OAK-10843: -- Summary: Flaky fullgc tests (was: Flaky between-cp tests) > Flaky fullgc tests > -- > > Key: OAK-10843 > URL: https://issues.apache.org/jira/browse/OAK-10843 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > As noted in OAK-10739 there is potential flakyness in tests with fullgc modes > "BETWEEN_CHECKPOINTS". This ticket is to look into fixing it. First measure > could be to disable them until we have a fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10739) Provide Support for Full Garbage Collection in Mongo Document Store
[ https://issues.apache.org/jira/browse/OAK-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850350#comment-17850350 ] Stefan Egli commented on OAK-10739: --- One suggestion is to disable the flaky tests - eg done for VersionGarbageCollectorIT in OAK-10843 > Provide Support for Full Garbage Collection in Mongo Document Store > --- > > Key: OAK-10739 > URL: https://issues.apache.org/jira/browse/OAK-10739 > Project: Jackrabbit Oak > Issue Type: Epic >Reporter: Rishabh Daim >Assignee: Rishabh Daim >Priority: Major > Labels: DetailedGC > Attachments: > org.apache.jackrabbit.oak.plugins.document.BranchCommitGCTest.txt, > org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorIT.txt > > > We need to provide the support to collect & remove the full garbage for > DocumentNodeStore. > At the time of creating this epic garbage includes orphaned nodes, deleted > properties, unmerged branch commits, and old revisions. > > This list can be updated in case a new type of garbage is found. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10843) Flaky between-cp tests
[ https://issues.apache.org/jira/browse/OAK-10843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850348#comment-17850348 ] Stefan Egli commented on OAK-10843: --- Was meant to create a PR first.but it got straight to trunk - https://github.com/apache/jackrabbit-oak/commit/e488628c018a08ad40c8f36281a076b1ef2dd9a6 > Flaky between-cp tests > -- > > Key: OAK-10843 > URL: https://issues.apache.org/jira/browse/OAK-10843 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Stefan Egli >Priority: Major > > As noted in OAK-10739 there is potential flakyness in tests with fullgc modes > "BETWEEN_CHECKPOINTS". This ticket is to look into fixing it. First measure > could be to disable them until we have a fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (OAK-10843) Flaky between-cp tests
Stefan Egli created OAK-10843: - Summary: Flaky between-cp tests Key: OAK-10843 URL: https://issues.apache.org/jira/browse/OAK-10843 Project: Jackrabbit Oak Issue Type: Task Components: documentmk Reporter: Stefan Egli As noted in OAK-10739 there is potential flakyness in tests with fullgc modes "BETWEEN_CHECKPOINTS". This ticket is to look into fixing it. First measure could be to disable them until we have a fix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10739) Provide Support for Full Garbage Collection in Mongo Document Store
[ https://issues.apache.org/jira/browse/OAK-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846348#comment-17846348 ] Stefan Egli commented on OAK-10739: --- Yes it is for now only implemented for MongoDB. The largest part is outside of DocumentStore though, so adding RDB support is a relatively small effort (compared to the actual feature) > Provide Support for Full Garbage Collection in Mongo Document Store > --- > > Key: OAK-10739 > URL: https://issues.apache.org/jira/browse/OAK-10739 > Project: Jackrabbit Oak > Issue Type: Epic >Reporter: Rishabh Daim >Assignee: Rishabh Daim >Priority: Major > Labels: DetailedGC > > We need to provide the support to collect & remove the full garbage for > DocumentNodeStore. > At the time of creating this epic garbage includes orphaned nodes, deleted > properties, unmerged branch commits, and old revisions. > > This list can be updated in case a new type of garbage is found. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OAK-10792) Rename DetailedGC to FullGC
[ https://issues.apache.org/jira/browse/OAK-10792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846330#comment-17846330 ] Stefan Egli commented on OAK-10792: --- Also added https://github.com/apache/jackrabbit-oak/pull/1458 > Rename DetailedGC to FullGC > --- > > Key: OAK-10792 > URL: https://issues.apache.org/jira/browse/OAK-10792 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core >Reporter: Daniel Iancu >Assignee: Rishabh Daim >Priority: Minor > Labels: DetailedGC > > Switching to FullGC instead of DetailedGC everywhere, method names, > constants, arguments etc -- This message was sent by Atlassian Jira (v8.20.10#820010)
