[jira] [Commented] (OAK-4439) Fix the errors reported by the Javadoc tool in JDK8
[ https://issues.apache.org/jira/browse/OAK-4439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320115#comment-15320115 ] Julian Reschke commented on OAK-4439: - Backported changes for RDB*: 1.4: http://svn.apache.org/r1747354 1.2: http://svn.apache.org/r1747355 1.0: http://svn.apache.org/r1747356 > Fix the errors reported by the Javadoc tool in JDK8 > --- > > Key: OAK-4439 > URL: https://issues.apache.org/jira/browse/OAK-4439 > Project: Jackrabbit Oak > Issue Type: Bug >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.6, 1.5.4 > > > Some Javadoc is not strict enough according to the Javadoc tool shipped in > JDK8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4200) [BlobGC] Improve collection times of blobs available
[ https://issues.apache.org/jira/browse/OAK-4200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320071#comment-15320071 ] Thomas Mueller commented on OAK-4200: - Some statistics (so that we don't forget): GC was run against an S3 datastore, without the improvements of this issue (also excluding OAK-4429 and OAK-4430), and took around 37 hours. 99.91% of that time was "retrieve of all blobs" (about 2 million). Retrieving references took about 1 minute 45 seconds, and sorting and deleting took around 10 seconds. > [BlobGC] Improve collection times of blobs available > > > Key: OAK-4200 > URL: https://issues.apache.org/jira/browse/OAK-4200 > Project: Jackrabbit Oak > Issue Type: Improvement >Reporter: Amit Jain >Assignee: Amit Jain > Fix For: 1.5.4 > > > The blob collection phase (Identifying all the blobs available in the data > store) is quite an expensive part of the whole GC process, taking up a few > hours sometimes on large repositories, due to iteration of the sub-folders in > the data store. > In an offline discussion with [~tmueller] and [~chetanm], the idea came up > that this phase can be faster if > * Blobs ids are tracked when the blobs are added for e.g. in a simple file > in the datastore per cluster node. > * GC then consolidates this file from all the cluster nodes and uses it to > get the candidates for GC. > * This variant of the MarkSweepGC can be triggered more frequently. It would > be ok to miss blob id additions to this file during a crash etc., as these > blobs can be cleaned up in the *regular* MarkSweepGC cycles triggered > occasionally. > We also may be able to track other metadata along with the blob ids like > paths, timestamps etc. for auditing/analytics, in-conjunction with OAK-3140. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4432) Ignore files in the root directory of the FileDataStore in #getAllIdentifiers
[ https://issues.apache.org/jira/browse/OAK-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain resolved OAK-4432. Resolution: Fixed Fix Version/s: 1.4.4 1.2.16 1.5.3 Fixed with: * trunk - http://svn.apache.org/viewvc?rev=1746981&view=rev * 1.4 - http://svn.apache.org/viewvc?rev=1747346&view=rev * 1.2 - http://svn.apache.org/viewvc?rev=1747348&view=rev > Ignore files in the root directory of the FileDataStore in #getAllIdentifiers > - > > Key: OAK-4432 > URL: https://issues.apache.org/jira/browse/OAK-4432 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Minor > Fix For: 1.6, 1.5.3, 1.2.16, 1.4.4 > > > The call to OakFileDataStore#getAllIdentifiers should ignore the the files > directly at the root of the DataStore (These files are used for > SharedDataStore etc). This does not cause any functional problems but leads > to logging warning in the logs. > There is already a check but it fails when the data store root is specified > as a relative path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4432) Ignore files in the root directory of the FileDataStore in #getAllIdentifiers
[ https://issues.apache.org/jira/browse/OAK-4432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain updated OAK-4432: --- Labels: (was: candidate_oak_1_2 candidate_oak_1_4) > Ignore files in the root directory of the FileDataStore in #getAllIdentifiers > - > > Key: OAK-4432 > URL: https://issues.apache.org/jira/browse/OAK-4432 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob >Reporter: Amit Jain >Assignee: Amit Jain >Priority: Minor > Fix For: 1.6, 1.5.3, 1.2.16, 1.4.4 > > > The call to OakFileDataStore#getAllIdentifiers should ignore the the files > directly at the root of the DataStore (These files are used for > SharedDataStore etc). This does not cause any functional problems but leads > to logging warning in the logs. > There is already a check but it fails when the data store root is specified > as a relative path. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4421) Optimize Revison fromString and toString implementation
[
https://issues.apache.org/jira/browse/OAK-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-4421.
--
Resolution: Fixed
Fix Version/s: 1.5.4
Applied the updated patch
* trunk - 1747347
> Optimize Revison fromString and toString implementation
> ---
>
> Key: OAK-4421
> URL: https://issues.apache.org/jira/browse/OAK-4421
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: performance
> Fix For: 1.6, 1.5.4
>
> Attachments: OAK-4421-v1.patch, OAK-4421-v2.patch, OAK-4421-v3.patch,
> hot-methods.png, revision-fromstring-caller.png
>
>
> Current implementation of Revision {{fromString}} and {{toString}} make use
> of std JDK API to perform string manipulation. While running some performance
> test it was seen that these 2 methods are called quite frequently and that
> adds up to some decent times. Further they also generate quite a bit of short
> lived objects.
> !hot-methods.png!
> It would be worthwhile to perform a micro benchmark of these method and
> optimize them further such that they perform better and also generate less
> garbage. The micro optimized code would be bit more complex but if
> performance numbers are better we can look into changing the current
> implementation
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4421) Optimize Revison fromString and toString implementation
[
https://issues.apache.org/jira/browse/OAK-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15319995#comment-15319995
]
Chetan Mehrotra commented on OAK-4421:
--
Updated benchmark results
{noformat}
Benchmark Mode Cnt Score
Error Units
RevisionFromStringBenchmark.revisionFromStringDefault thrpt 10 5430745.612
± 1113462.888 ops/s
RevisionFromStringBenchmark.revisionFromString_v1 thrpt 10 31371073.348
± 3042813.504 ops/s
RevisionFromStringBenchmark.revisionFromString_v3 thrpt 10 28943711.612
± 1409866.424 ops/s
Benchmark Mode Cnt Score
Error Units
RevisionAsStringBenchmark.revisionAsStringDefault thrpt 10 9265267.138 ±
175378.646 ops/s
RevisionAsStringBenchmark.revisionAsString_v1 thrpt 10 10247322.408 ±
826041.526 ops/s
RevisionAsStringBenchmark.revisionAsString_v3 thrpt 10 11868436.063 ±
538479.238 ops/s
{noformat}
> Optimize Revison fromString and toString implementation
> ---
>
> Key: OAK-4421
> URL: https://issues.apache.org/jira/browse/OAK-4421
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-4421-v1.patch, OAK-4421-v2.patch, OAK-4421-v3.patch,
> hot-methods.png, revision-fromstring-caller.png
>
>
> Current implementation of Revision {{fromString}} and {{toString}} make use
> of std JDK API to perform string manipulation. While running some performance
> test it was seen that these 2 methods are called quite frequently and that
> adds up to some decent times. Further they also generate quite a bit of short
> lived objects.
> !hot-methods.png!
> It would be worthwhile to perform a micro benchmark of these method and
> optimize them further such that they perform better and also generate less
> garbage. The micro optimized code would be bit more complex but if
> performance numbers are better we can look into changing the current
> implementation
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4429) [oak-blob-cloud] S3Backend#getAllIdentifiers should not store all elements in memory
[ https://issues.apache.org/jira/browse/OAK-4429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain resolved OAK-4429. Resolution: Fixed Fix Version/s: 1.5.4 In trunk with http://svn.apache.org/viewvc?rev=1747341&view=rev > [oak-blob-cloud] S3Backend#getAllIdentifiers should not store all elements in > memory > > > Key: OAK-4429 > URL: https://issues.apache.org/jira/browse/OAK-4429 > Project: Jackrabbit Oak > Issue Type: Bug > Components: blob >Reporter: Amit Jain >Assignee: Amit Jain > Labels: candidate_oak_1_2, candidate_oak_1_4 > Fix For: 1.6, 1.5.4 > > > While fetching all blob ids from S3 the data is stored on memory before > returning an iterator over it. This can be problematic when the number of > blobs stored S3 are in millions. > The code should be changed to store the elements on a temp file and then > returning an iterator over it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4430) DataStoreBlobStore#getAllChunkIds fetches DataRecord when not needed
[
https://issues.apache.org/jira/browse/OAK-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain resolved OAK-4430.
Resolution: Fixed
Fix Version/s: 1.5.4
In trunk with http://svn.apache.org/viewvc?rev=1747342&view=rev
> DataStoreBlobStore#getAllChunkIds fetches DataRecord when not needed
>
>
> Key: OAK-4430
> URL: https://issues.apache.org/jira/browse/OAK-4430
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob
>Reporter: Amit Jain
>Assignee: Amit Jain
> Labels: candidate_oak_1_2, candidate_oak_1_4
> Fix For: 1.6, 1.5.4
>
>
> DataStoreBlobStore#getAllChunkIds loads the DataRecord for checking that the
> lastModifiedTime criteria is satisfied against the given
> {{maxLastModifiedTime}}.
> When the {{maxLastModifiedTime}} has a value 0 it effectively means ignore
> any last modified time check (and which is the only usage currently from
> MarkSweepGarbageCollector). This should ignore fetching the DataRecords as
> this can be very expensive for e.g on calls to S3 with millions of blobs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4435) checkpointDeduplicationTest sometimes fails on Jenkins
[
https://issues.apache.org/jira/browse/OAK-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig reassigned OAK-4435:
--
Assignee: Michael Dürig
> checkpointDeduplicationTest sometimes fails on Jenkins
> --
>
> Key: OAK-4435
> URL: https://issues.apache.org/jira/browse/OAK-4435
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: compaction, gc, test
> Fix For: 1.6
>
>
> {{CompactionAndCleanupIT.checkpointDeduplication}} irregularly
> [fails|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/938/jdk=latest1.7,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/console]
> on Jenkins.
> This might point to an issue with the de-duplication caches, which are
> crucial in getting the checkpoints de-duplicated.
> {code}
> checkpointDeduplicationTest(org.apache.jackrabbit.oak.segment.CompactionAndCleanupIT)
> Time elapsed: 0.15 sec <<< FAILURE!
> org.junit.ComparisonFailure:
> expected:<[7211975a-04ce-45ff-aff5-16795ec2cc72]:261932> but
> was:<[11083c4b-9b2e-4d17-a8c0-8f6b1f2a3173]:261932>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.jackrabbit.oak.segment.CompactionAndCleanupIT.checkpointDeduplicationTest(CompactionAndCleanupIT.java:899)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4127) Cleanup creates new generation of tar file without removing any segments
[ https://issues.apache.org/jira/browse/OAK-4127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig resolved OAK-4127. Resolution: Not A Problem Resolving as not a problem as this has been most likely caused by OAK-4147 > Cleanup creates new generation of tar file without removing any segments > - > > Key: OAK-4127 > URL: https://issues.apache.org/jira/browse/OAK-4127 > Project: Jackrabbit Oak > Issue Type: Bug > Components: segment-tar >Reporter: Michael Dürig >Priority: Minor > Labels: cleanup, gc > Fix For: 1.6 > > > On some deployments I have seen tar files with a quite hight generation > post-fix (e.g. 'v'). From the log files I could deduce that this particular > tar file was rewritten multiple times without actually any segment being > removed. > I assume this is caused by the 25% [gain threshold | > https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment/src/main/java/org/apache/jackrabbit/oak/plugins/segment/file/TarReader.java#L789] > not taking the sizes contributed by the index and the graph entries into > account. > We should try to come up with a test case validating above hypothesis. A fix > should then be relatively straight forward: either include the sizes of these > two entries in the calculation or skip further clean cycles if a file size > drops below a certain size. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-1553) More sophisticated conflict resolution when concurrently adding nodes
[
https://issues.apache.org/jira/browse/OAK-1553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-1553:
---
Flagged: (was: Impediment)
> More sophisticated conflict resolution when concurrently adding nodes
> -
>
> Key: OAK-1553
> URL: https://issues.apache.org/jira/browse/OAK-1553
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, mongomk, segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: concurrency, technical_debt
> Attachments: OAK-1553.patch
>
>
> {{MicroKernel.rebase}} currently specifies: "addExistingNode: A node has been
> added that is different from a node of them same name that has been added to
> the trunk."
> This is somewhat troublesome in the case where the same node with different
> but non conflicting child items is added concurrently:
> {code}
> f.add("fo").add("u1"); commit();
> f.add("fo").add("u2"); commit();
> {code}
> currently fails with a conflict because {{fo}} is not the same node for the
> both cases. See discussion http://markmail.org/message/flst4eiqvbp4gi3z
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-1553) More sophisticated conflict resolution when concurrently adding nodes
[
https://issues.apache.org/jira/browse/OAK-1553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-1553:
---
Flagged: Impediment
> More sophisticated conflict resolution when concurrently adding nodes
> -
>
> Key: OAK-1553
> URL: https://issues.apache.org/jira/browse/OAK-1553
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core, mongomk, segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: concurrency, technical_debt
> Attachments: OAK-1553.patch
>
>
> {{MicroKernel.rebase}} currently specifies: "addExistingNode: A node has been
> added that is different from a node of them same name that has been added to
> the trunk."
> This is somewhat troublesome in the case where the same node with different
> but non conflicting child items is added concurrently:
> {code}
> f.add("fo").add("u1"); commit();
> f.add("fo").add("u2"); commit();
> {code}
> currently fails with a conflict because {{fo}} is not the same node for the
> both cases. See discussion http://markmail.org/message/flst4eiqvbp4gi3z
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3967) FailoverIPRangeTest failing sporadically
[ https://issues.apache.org/jira/browse/OAK-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-3967: - Fix Version/s: 1.6 > FailoverIPRangeTest failing sporadically > > > Key: OAK-3967 > URL: https://issues.apache.org/jira/browse/OAK-3967 > Project: Jackrabbit Oak > Issue Type: Bug > Components: tarmk-standby > Environment: See also > https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/721/ >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu >Priority: Minor > Fix For: 1.6 > > > I'll disable the test for now to have a stable build. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4083) Simplify concurrency when loading data from the primary
[
https://issues.apache.org/jira/browse/OAK-4083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4083:
-
Fix Version/s: 1.2.16
> Simplify concurrency when loading data from the primary
> ---
>
> Key: OAK-4083
> URL: https://issues.apache.org/jira/browse/OAK-4083
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: tarmk-standby
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.4.1, 1.5.0, 1.2.16
>
> Attachments: OAK-4083-01.patch
>
>
> The cold standby client is overly complicated and potentially buggy when
> segments are loaded from the primary.
> In particular:
> - when the {{StandbyClientHandler}} is removed from the pipeline, its
> {{EventExecutorGroup}} is passed to the {{SegmentLoaderHandler}} to signal a
> surrogate of the "channel close" event. It would be more sensible for
> {{SegmentLoaderHandler}} to listen to the proper "channel close" event
> instead.
> - after the {{SegmentLoaderHandler}} is added to the pipeline, the "channel
> active" callback method is called directly using the
> {{ChannelHandlerContext}} of the {{StandbyClientHandler}}. It would be better
> for {{SegmentLoaderHandler}} to use a proper initialisation point like, in
> example, the "handler added" callback method. The {{SegmentLoaderHandler}}
> should also use its own {{ChannelHandlerContext}}, instead of owning one from
> another handler.
> - {{SegmentLoaderHandler}} saves and uses the {{ChannelHandlerContext}} of
> the {{StandbyClientHandler}}. The only purpose for it is to use the
> {{EventExecutorGroup}} of the {{StandbyClientHandler}} to issue "write
> messages" events. The {{SegmentLoaderHandler}} should use Netty's event loop
> to issue I/O operations, and instead use a different thread for business
> logic - in this case, the synchronization process between primary and standby
> instances.
> - The {{StandbyClientHandler}} is registered with its own
> {{EventExecutorGroup}} with four threads. This is overkill, since the
> synchronization process should run anyway in a separate thread (see below).
> The {{StandbyClientHandler}} should use the default event loop.
> - The {{SegmentLoaderHandler}} is registered with its own
> {{EventExecutorGroup}} with four threads. This is overkill, since
> request/response cycle between the standby and the primary instances is
> synchronous. The {{SegmentLoaderHandler}} should instead use the default
> event loop for I/O events and run another (single) thread to execute the
> synchronization process.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4083) Simplify concurrency when loading data from the primary
[
https://issues.apache.org/jira/browse/OAK-4083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318989#comment-15318989
]
Alex Parvulescu commented on OAK-4083:
--
merged to 1.2 with r1747284
> Simplify concurrency when loading data from the primary
> ---
>
> Key: OAK-4083
> URL: https://issues.apache.org/jira/browse/OAK-4083
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: tarmk-standby
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.4.1, 1.5.0, 1.2.16
>
> Attachments: OAK-4083-01.patch
>
>
> The cold standby client is overly complicated and potentially buggy when
> segments are loaded from the primary.
> In particular:
> - when the {{StandbyClientHandler}} is removed from the pipeline, its
> {{EventExecutorGroup}} is passed to the {{SegmentLoaderHandler}} to signal a
> surrogate of the "channel close" event. It would be more sensible for
> {{SegmentLoaderHandler}} to listen to the proper "channel close" event
> instead.
> - after the {{SegmentLoaderHandler}} is added to the pipeline, the "channel
> active" callback method is called directly using the
> {{ChannelHandlerContext}} of the {{StandbyClientHandler}}. It would be better
> for {{SegmentLoaderHandler}} to use a proper initialisation point like, in
> example, the "handler added" callback method. The {{SegmentLoaderHandler}}
> should also use its own {{ChannelHandlerContext}}, instead of owning one from
> another handler.
> - {{SegmentLoaderHandler}} saves and uses the {{ChannelHandlerContext}} of
> the {{StandbyClientHandler}}. The only purpose for it is to use the
> {{EventExecutorGroup}} of the {{StandbyClientHandler}} to issue "write
> messages" events. The {{SegmentLoaderHandler}} should use Netty's event loop
> to issue I/O operations, and instead use a different thread for business
> logic - in this case, the synchronization process between primary and standby
> instances.
> - The {{StandbyClientHandler}} is registered with its own
> {{EventExecutorGroup}} with four threads. This is overkill, since the
> synchronization process should run anyway in a separate thread (see below).
> The {{StandbyClientHandler}} should use the default event loop.
> - The {{SegmentLoaderHandler}} is registered with its own
> {{EventExecutorGroup}} with four threads. This is overkill, since
> request/response cycle between the standby and the primary instances is
> synchronous. The {{SegmentLoaderHandler}} should instead use the default
> event loop for I/O events and run another (single) thread to execute the
> synchronization process.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4083) Simplify concurrency when loading data from the primary
[
https://issues.apache.org/jira/browse/OAK-4083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4083:
-
Fix Version/s: 1.4.1
> Simplify concurrency when loading data from the primary
> ---
>
> Key: OAK-4083
> URL: https://issues.apache.org/jira/browse/OAK-4083
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: tarmk-standby
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.4.1, 1.5.0
>
> Attachments: OAK-4083-01.patch
>
>
> The cold standby client is overly complicated and potentially buggy when
> segments are loaded from the primary.
> In particular:
> - when the {{StandbyClientHandler}} is removed from the pipeline, its
> {{EventExecutorGroup}} is passed to the {{SegmentLoaderHandler}} to signal a
> surrogate of the "channel close" event. It would be more sensible for
> {{SegmentLoaderHandler}} to listen to the proper "channel close" event
> instead.
> - after the {{SegmentLoaderHandler}} is added to the pipeline, the "channel
> active" callback method is called directly using the
> {{ChannelHandlerContext}} of the {{StandbyClientHandler}}. It would be better
> for {{SegmentLoaderHandler}} to use a proper initialisation point like, in
> example, the "handler added" callback method. The {{SegmentLoaderHandler}}
> should also use its own {{ChannelHandlerContext}}, instead of owning one from
> another handler.
> - {{SegmentLoaderHandler}} saves and uses the {{ChannelHandlerContext}} of
> the {{StandbyClientHandler}}. The only purpose for it is to use the
> {{EventExecutorGroup}} of the {{StandbyClientHandler}} to issue "write
> messages" events. The {{SegmentLoaderHandler}} should use Netty's event loop
> to issue I/O operations, and instead use a different thread for business
> logic - in this case, the synchronization process between primary and standby
> instances.
> - The {{StandbyClientHandler}} is registered with its own
> {{EventExecutorGroup}} with four threads. This is overkill, since the
> synchronization process should run anyway in a separate thread (see below).
> The {{StandbyClientHandler}} should use the default event loop.
> - The {{SegmentLoaderHandler}} is registered with its own
> {{EventExecutorGroup}} with four threads. This is overkill, since
> request/response cycle between the standby and the primary instances is
> synchronous. The {{SegmentLoaderHandler}} should instead use the default
> event loop for I/O events and run another (single) thread to execute the
> synchronization process.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4058) Cold Standby intermittently throws RejectedExecutionException
[
https://issues.apache.org/jira/browse/OAK-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-4058:
-
Fix Version/s: 1.2.16
> Cold Standby intermittently throws RejectedExecutionException
> -
>
> Key: OAK-4058
> URL: https://issues.apache.org/jira/browse/OAK-4058
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: tarmk-standby
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.4, 1.4.0, 1.2.16
>
> Attachments: OAK-4058.patch
>
>
> The Cold Standby subsystem very frequently catches and logs
> {{RejectedExecutionException}} instances thrown by Netty. These exceptions
> are most likely caused by an incorrect usage of the thread pools allocated by
> Netty.
> {noformat}
> 25.02.2016 09:23:02.517 *ERROR* [defaultEventExecutorGroup-1306-1]
> org.apache.jackrabbit.oak.plugins.segment.standby.client.SegmentLoaderHandler
> Exception caught, closing channel.
> java.util.concurrent.RejectedExecutionException: event executor terminated
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:745)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:322)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:728)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:221)
> at
> io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233)
> at
> io.netty.channel.AbstractChannelHandlerContext.access$400(AbstractChannelHandlerContext.java:32)
> at
> io.netty.channel.AbstractChannelHandlerContext$5.run(AbstractChannelHandlerContext.java:224)
> at
> io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:36)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4058) Cold Standby intermittently throws RejectedExecutionException
[
https://issues.apache.org/jira/browse/OAK-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318979#comment-15318979
]
Alex Parvulescu commented on OAK-4058:
--
merged to 1.2 with r1747280
> Cold Standby intermittently throws RejectedExecutionException
> -
>
> Key: OAK-4058
> URL: https://issues.apache.org/jira/browse/OAK-4058
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: tarmk-standby
>Reporter: Francesco Mari
>Assignee: Francesco Mari
> Fix For: 1.4, 1.4.0, 1.2.16
>
> Attachments: OAK-4058.patch
>
>
> The Cold Standby subsystem very frequently catches and logs
> {{RejectedExecutionException}} instances thrown by Netty. These exceptions
> are most likely caused by an incorrect usage of the thread pools allocated by
> Netty.
> {noformat}
> 25.02.2016 09:23:02.517 *ERROR* [defaultEventExecutorGroup-1306-1]
> org.apache.jackrabbit.oak.plugins.segment.standby.client.SegmentLoaderHandler
> Exception caught, closing channel.
> java.util.concurrent.RejectedExecutionException: event executor terminated
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.reject(SingleThreadEventExecutor.java:745)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.addTask(SingleThreadEventExecutor.java:322)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:728)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:221)
> at
> io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:233)
> at
> io.netty.channel.AbstractChannelHandlerContext.access$400(AbstractChannelHandlerContext.java:32)
> at
> io.netty.channel.AbstractChannelHandlerContext$5.run(AbstractChannelHandlerContext.java:224)
> at
> io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:36)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3961) Cold Standby revisit timeout setup
[ https://issues.apache.org/jira/browse/OAK-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-3961: - Fix Version/s: 1.2.16 > Cold Standby revisit timeout setup > -- > > Key: OAK-3961 > URL: https://issues.apache.org/jira/browse/OAK-3961 > Project: Jackrabbit Oak > Issue Type: Bug > Components: tarmk-standby >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.4, 1.3.15, 1.2.16 > > > The timeout settings are too large and inefficient, making all the tests very > slow. On top of this the current timeout if being enforced in 2 places, which > turns out it doesn't play too well with the sync mechanism: > * one is via the _ReadTimeoutHandler_ in the _StandbyClient_ > * second is in the _SegmentLoaderHandler_ > as it turns out the first one is a global kill switch, and it will fail any > transaction larger than the set value (_all_ of the sync cycle), which is not > what I meant to do with it, so I'll remove it and only leave the second one, > which is a timeout per request (segment or binary). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3961) Cold Standby revisit timeout setup
[ https://issues.apache.org/jira/browse/OAK-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318952#comment-15318952 ] Alex Parvulescu commented on OAK-3961: -- merged to 1.2 branch with r1747277 > Cold Standby revisit timeout setup > -- > > Key: OAK-3961 > URL: https://issues.apache.org/jira/browse/OAK-3961 > Project: Jackrabbit Oak > Issue Type: Bug > Components: tarmk-standby >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.4, 1.3.15, 1.2.16 > > > The timeout settings are too large and inefficient, making all the tests very > slow. On top of this the current timeout if being enforced in 2 places, which > turns out it doesn't play too well with the sync mechanism: > * one is via the _ReadTimeoutHandler_ in the _StandbyClient_ > * second is in the _SegmentLoaderHandler_ > as it turns out the first one is a global kill switch, and it will fail any > transaction larger than the set value (_all_ of the sync cycle), which is not > what I meant to do with it, so I'll remove it and only leave the second one, > which is a timeout per request (segment or binary). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3961) Cold Standby revisit timeout setup
[ https://issues.apache.org/jira/browse/OAK-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-3961: - Labels: (was: candidate_oak_1_2) > Cold Standby revisit timeout setup > -- > > Key: OAK-3961 > URL: https://issues.apache.org/jira/browse/OAK-3961 > Project: Jackrabbit Oak > Issue Type: Bug > Components: tarmk-standby >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.4, 1.3.15, 1.2.16 > > > The timeout settings are too large and inefficient, making all the tests very > slow. On top of this the current timeout if being enforced in 2 places, which > turns out it doesn't play too well with the sync mechanism: > * one is via the _ReadTimeoutHandler_ in the _StandbyClient_ > * second is in the _SegmentLoaderHandler_ > as it turns out the first one is a global kill switch, and it will fail any > transaction larger than the set value (_all_ of the sync cycle), which is not > what I meant to do with it, so I'll remove it and only leave the second one, > which is a timeout per request (segment or binary). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4278) Fix backup and restore
[
https://issues.apache.org/jira/browse/OAK-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4278:
---
Priority: Critical (was: Blocker)
> Fix backup and restore
> --
>
> Key: OAK-4278
> URL: https://issues.apache.org/jira/browse/OAK-4278
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Alex Parvulescu
>Priority: Critical
> Labels: backup, restore, technical_debt
> Fix For: 1.6
>
> Attachments: OAK-4278-v0.patch
>
>
> {{FileStoreBackup}} and {{FileStoreRestore}} are currently broken as a side
> effect of the fix from OAK-3348. ({{FileStoreBackupTest}} is currently being
> skipped).
> In {{oak-segment}} backup and restore functionality relied on the
> {{Compactor}} class. The latter is gone in {{oak-segment-next}} as it is not
> needed for online compaction any more.
> Instead of sharing functionality from compaction directly, I think backup and
> restore should come up with its own implementation that could be individually
> tweaked for its task. If there is commonalities with offline compaction those
> can still be shared thorough a common base class. See OAK-4279
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4153) segment's compareAgainstBaseState wont call childNodeDeleted when deleting last and adding n nodes
[
https://issues.apache.org/jira/browse/OAK-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4153:
---
Fix Version/s: 1.5.1
> segment's compareAgainstBaseState wont call childNodeDeleted when deleting
> last and adding n nodes
> --
>
> Key: OAK-4153
> URL: https://issues.apache.org/jira/browse/OAK-4153
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Affects Versions: 1.2.13, 1.0.29, 1.4.1, 1.5.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Critical
> Fix For: 1.5.1
>
> Attachments: OAK-4153-2.patch, OAK-4153-3.patch, OAK-4153.patch,
> OAK-4153.simplified.patch
>
>
> {{SegmentNodeState.compareAgainstBaseState}} fails to call
> {{NodeStateDiff.childNodeDeleted}} when for the same parent the only child is
> deleted and at the same time multiple new, different children are added.
> Reason is that the [current
> code|https://github.com/apache/jackrabbit-oak/blob/a9ce70b61567ffe27529dad8eb5d38ced77cf8ad/oak-segment/src/main/java/org/apache/jackrabbit/oak/plugins/segment/SegmentNodeState.java#L558]
> for '{{afterChildName == MANY_CHILD_NODES}}' *and* '{{beforeChildName ==
> ONE_CHILD_NODE}}' does not handle all cases: it assumes that 'after' contains
> the 'before' child and doesn't handle the situation where the 'before' child
> has gone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4153) segment's compareAgainstBaseState wont call childNodeDeleted when deleting last and adding n nodes
[
https://issues.apache.org/jira/browse/OAK-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4153:
---
Fix Version/s: (was: 1.6)
> segment's compareAgainstBaseState wont call childNodeDeleted when deleting
> last and adding n nodes
> --
>
> Key: OAK-4153
> URL: https://issues.apache.org/jira/browse/OAK-4153
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
>Affects Versions: 1.2.13, 1.0.29, 1.4.1, 1.5.0
>Reporter: Stefan Egli
>Assignee: Stefan Egli
>Priority: Critical
> Attachments: OAK-4153-2.patch, OAK-4153-3.patch, OAK-4153.patch,
> OAK-4153.simplified.patch
>
>
> {{SegmentNodeState.compareAgainstBaseState}} fails to call
> {{NodeStateDiff.childNodeDeleted}} when for the same parent the only child is
> deleted and at the same time multiple new, different children are added.
> Reason is that the [current
> code|https://github.com/apache/jackrabbit-oak/blob/a9ce70b61567ffe27529dad8eb5d38ced77cf8ad/oak-segment/src/main/java/org/apache/jackrabbit/oak/plugins/segment/SegmentNodeState.java#L558]
> for '{{afterChildName == MANY_CHILD_NODES}}' *and* '{{beforeChildName ==
> ONE_CHILD_NODE}}' does not handle all cases: it assumes that 'after' contains
> the 'before' child and doesn't handle the situation where the 'before' child
> has gone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4292) Document Oak segment-tar
[ https://issues.apache.org/jira/browse/OAK-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4292: --- Description: Document Oak Segment Tar. Specifically: * New and changed configuration and monitoring options * Changes in gc (OAK-3348 et. all) * Changes in segment / tar format (OAK-3348) was: Document Oak Segment Next. Specifically: * New and changed configuration and monitoring options * Changes in gc (OAK-3348 et. all) * Changes in segment / tar format (OAK-3348) > Document Oak segment-tar > > > Key: OAK-4292 > URL: https://issues.apache.org/jira/browse/OAK-4292 > Project: Jackrabbit Oak > Issue Type: Task > Components: doc, segment-tar >Reporter: Michael Dürig > Labels: documentation, gc > Fix For: 1.6 > > > Document Oak Segment Tar. Specifically: > * New and changed configuration and monitoring options > * Changes in gc (OAK-3348 et. all) > * Changes in segment / tar format (OAK-3348) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4292) Document Oak segment-tar
[ https://issues.apache.org/jira/browse/OAK-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-4292: --- Summary: Document Oak segment-tar (was: Document Oak segment next) > Document Oak segment-tar > > > Key: OAK-4292 > URL: https://issues.apache.org/jira/browse/OAK-4292 > Project: Jackrabbit Oak > Issue Type: Task > Components: doc, segment-tar >Reporter: Michael Dürig > Labels: documentation, gc > Fix For: 1.6 > > > Document Oak Segment Next. Specifically: > * New and changed configuration and monitoring options > * Changes in gc (OAK-3348 et. all) > * Changes in segment / tar format (OAK-3348) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4435) checkpointDeduplicationTest sometimes fails on Jenkins
[
https://issues.apache.org/jira/browse/OAK-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318752#comment-15318752
]
Michael Dürig commented on OAK-4435:
This failure could have the same cause as the one of OAK-4436: the shared non
mutable default gc options instance.
Let's resolve this issue if the failure doesn't show up for a while.
> checkpointDeduplicationTest sometimes fails on Jenkins
> --
>
> Key: OAK-4435
> URL: https://issues.apache.org/jira/browse/OAK-4435
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Priority: Critical
> Labels: compaction, gc, test
> Fix For: 1.6
>
>
> {{CompactionAndCleanupIT.checkpointDeduplication}} irregularly
> [fails|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/938/jdk=latest1.7,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/console]
> on Jenkins.
> This might point to an issue with the de-duplication caches, which are
> crucial in getting the checkpoints de-duplicated.
> {code}
> checkpointDeduplicationTest(org.apache.jackrabbit.oak.segment.CompactionAndCleanupIT)
> Time elapsed: 0.15 sec <<< FAILURE!
> org.junit.ComparisonFailure:
> expected:<[7211975a-04ce-45ff-aff5-16795ec2cc72]:261932> but
> was:<[11083c4b-9b2e-4d17-a8c0-8f6b1f2a3173]:261932>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.jackrabbit.oak.segment.CompactionAndCleanupIT.checkpointDeduplicationTest(CompactionAndCleanupIT.java:899)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4285) Align cleanup of segment id tables with the new cleanup strategy
[ https://issues.apache.org/jira/browse/OAK-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318748#comment-15318748 ] Michael Dürig commented on OAK-4285: I just implemented the sketched approach in OAK-2403. See http://svn.apache.org/viewvc?rev=1747254&view=rev. I think there is no need for further action on the segment id tables and we should resolve this issue as won't fix. [~alex.parvulescu], WDYT? > Align cleanup of segment id tables with the new cleanup strategy > - > > Key: OAK-4285 > URL: https://issues.apache.org/jira/browse/OAK-4285 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-tar >Reporter: Michael Dürig >Assignee: Alex Parvulescu > Labels: cleanup, gc > Fix For: 1.6 > > Attachments: OAK_4285.patch, OAK_4285.patch > > > We need to align cleanup of the segment id tables with the new "brutal" > strategy introduced with OAK-3348. That is, we need to remove those segment > id's from the segment id tables whose segment have actually been gc'ed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2403) Improve monitoring capabilities for TarMk revision gc
[ https://issues.apache.org/jira/browse/OAK-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig resolved OAK-2403. Resolution: Fixed All subtasks are done, resolving as fixed. > Improve monitoring capabilities for TarMk revision gc > - > > Key: OAK-2403 > URL: https://issues.apache.org/jira/browse/OAK-2403 > Project: Jackrabbit Oak > Issue Type: Task > Components: segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig > Labels: gc, monitoring, tooling > Fix For: 1.6 > > > Container devoted to improving monitoring of the TarMk revision garbage > collection process. The overall goal is to make it more transparent what > revision gc does, how it performs, why it failed etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2405) Monitoring to track old NodeStates
[
https://issues.apache.org/jira/browse/OAK-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-2405.
Resolution: Fixed
Fixed at http://svn.apache.org/viewvc?rev=1747254&view=rev
With these changes a {{SNFE}} will cause logging of additional information (if
available) regarding the gc cycles that removed it:
{code}
Segment not found: 7e1a21a5-b74a-4688-a506-5c35ebc89d0b.
SegmentId
age=759ms,gc-count=3,timestamp=1465309323368,gc-status=success,store-generation=2,reclaim-predicate=(generation<=1)
{code}
> Monitoring to track old NodeStates
> --
>
> Key: OAK-2405
> URL: https://issues.apache.org/jira/browse/OAK-2405
> Project: Jackrabbit Oak
> Issue Type: Sub-task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: gc, monitoring, tooling
> Fix For: 1.6
>
>
> We should add some monitoring that allows us to track "old" node states,
> which potentially block revision gc.
> Possible approaches:
> * Add monitoring too old revisions (root node states) along with the stack
> traces from where they have been acquired.
> * Include RecordId of root node state in the {{SessionMBean}}.
> * Add additional tooling on top of the {{SessionMBean}} to make it easier to
> make sense of the wealth of information provided.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4436) HeavyWriteIT sporadically fails
[
https://issues.apache.org/jira/browse/OAK-4436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-4436.
Resolution: Fixed
Fixed at http://svn.apache.org/viewvc?rev=1747256&view=rev
The problem was the "mutable" constant for default segment gc options, which
was shared between different stores.
> HeavyWriteIT sporadically fails
> ---
>
> Key: OAK-4436
> URL: https://issues.apache.org/jira/browse/OAK-4436
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: gc
> Fix For: 1.6
>
>
> I've seen {{HeavyWriteIT}} fail sporadically on my local checkout.
> {noformat}
> 3d13e2927fc0d75454a692ef5c8703880dc2ea0d
> org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
> 31b75992-aaf7-4f2b-a5de-b5a268c1fdb3 not found
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$14.call(FileStore.java:1377)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$14.call(FileStore.java:1317)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1011)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:972)
> at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:283)
> at
> org.apache.jackrabbit.oak.segment.SegmentCache.geSegment(SegmentCache.java:80)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1317)
> at
> org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:111)
> at
> org.apache.jackrabbit.oak.segment.RecordId.getSegment(RecordId.java:94)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.isOldGeneration(SegmentWriter.java:1010)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNodeUncached(SegmentWriter.java:906)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(SegmentWriter.java:885)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.access$700(SegmentWriter.java:319)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter$8.execute(SegmentWriter.java:277)
> at
> org.apache.jackrabbit.oak.segment.SegmentBufferWriterPool.execute(SegmentBufferWriterPool.java:110)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter.writeNode(SegmentWriter.java:274)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:111)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.(SegmentNodeStore.java:516)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:284)
> at
> org.apache.jackrabbit.oak.segment.HeavyWriteIT.heavyWrite(HeavyWriteIT.java:91)
> {noformat}
> I suspect this is a problem with {{isOldGeneration}} itself not being
> prepared for the old segment actually being gone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info
[
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek resolved OAK-4420.
Resolution: Fixed
> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.4
>
> Attachments: OAK-4420-naive.patch, OAK-4420.patch
>
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take
> care of moving the checkpoint data and meta. This will save a very expensive
> full-reindex.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info
[
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4420:
---
Fix Version/s: 1.5.4
> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.4
>
> Attachments: OAK-4420-naive.patch, OAK-4420.patch
>
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take
> care of moving the checkpoint data and meta. This will save a very expensive
> full-reindex.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info
[
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318649#comment-15318649
]
Tomek Rękawek commented on OAK-4420:
I've followed [~mduerig] advice and used the FileStore API. Also, I've added
test confirming that the data are de-duplicated.
Fixed for trunk in [r1747251|https://svn.apache.org/r1747251].
> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.4
>
> Attachments: OAK-4420-naive.patch, OAK-4420.patch
>
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take
> care of moving the checkpoint data and meta. This will save a very expensive
> full-reindex.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-4436) HeavyWriteIT sporadically fails
[
https://issues.apache.org/jira/browse/OAK-4436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig reassigned OAK-4436:
--
Assignee: Michael Dürig
> HeavyWriteIT sporadically fails
> ---
>
> Key: OAK-4436
> URL: https://issues.apache.org/jira/browse/OAK-4436
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
>Priority: Critical
> Labels: gc
> Fix For: 1.6
>
>
> I've seen {{HeavyWriteIT}} fail sporadically on my local checkout.
> {noformat}
> 3d13e2927fc0d75454a692ef5c8703880dc2ea0d
> org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
> 31b75992-aaf7-4f2b-a5de-b5a268c1fdb3 not found
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$14.call(FileStore.java:1377)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore$14.call(FileStore.java:1317)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1011)
> at
> org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:972)
> at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:283)
> at
> org.apache.jackrabbit.oak.segment.SegmentCache.geSegment(SegmentCache.java:80)
> at
> org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1317)
> at
> org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:111)
> at
> org.apache.jackrabbit.oak.segment.RecordId.getSegment(RecordId.java:94)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.isOldGeneration(SegmentWriter.java:1010)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNodeUncached(SegmentWriter.java:906)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(SegmentWriter.java:885)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.access$700(SegmentWriter.java:319)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter$8.execute(SegmentWriter.java:277)
> at
> org.apache.jackrabbit.oak.segment.SegmentBufferWriterPool.execute(SegmentBufferWriterPool.java:110)
> at
> org.apache.jackrabbit.oak.segment.SegmentWriter.writeNode(SegmentWriter.java:274)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:111)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.(SegmentNodeStore.java:516)
> at
> org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:284)
> at
> org.apache.jackrabbit.oak.segment.HeavyWriteIT.heavyWrite(HeavyWriteIT.java:91)
> {noformat}
> I suspect this is a problem with {{isOldGeneration}} itself not being
> prepared for the old segment actually being gone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3963) Cold Standby optimize sync for checkpoints
[ https://issues.apache.org/jira/browse/OAK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318644#comment-15318644 ] Alex Parvulescu commented on OAK-3963: -- merged to 1.2 with r1747250 > Cold Standby optimize sync for checkpoints > -- > > Key: OAK-3963 > URL: https://issues.apache.org/jira/browse/OAK-3963 > Project: Jackrabbit Oak > Issue Type: Bug > Components: tarmk-standby >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.4, 1.3.15, 1.2.16 > > > Checkpoint sync doesn't need to rely on a full traversal for binary > verification, as referenced binaries are probably already there in the > datastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3963) Cold Standby optimize sync for checkpoints
[ https://issues.apache.org/jira/browse/OAK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-3963: - Fix Version/s: 1.2.16 > Cold Standby optimize sync for checkpoints > -- > > Key: OAK-3963 > URL: https://issues.apache.org/jira/browse/OAK-3963 > Project: Jackrabbit Oak > Issue Type: Bug > Components: tarmk-standby >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.4, 1.3.15, 1.2.16 > > > Checkpoint sync doesn't need to rely on a full traversal for binary > verification, as referenced binaries are probably already there in the > datastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3963) Cold Standby optimize sync for checkpoints
[ https://issues.apache.org/jira/browse/OAK-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-3963: - Labels: (was: candidate_oak_1_2) > Cold Standby optimize sync for checkpoints > -- > > Key: OAK-3963 > URL: https://issues.apache.org/jira/browse/OAK-3963 > Project: Jackrabbit Oak > Issue Type: Bug > Components: tarmk-standby >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.4, 1.3.15, 1.2.16 > > > Checkpoint sync doesn't need to rely on a full traversal for binary > verification, as referenced binaries are probably already there in the > datastore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3794) The Cold Standby should expect loops in the segment graph
[ https://issues.apache.org/jira/browse/OAK-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318639#comment-15318639 ] Alex Parvulescu commented on OAK-3794: -- merged to 1.2 branch with r1747249 > The Cold Standby should expect loops in the segment graph > - > > Key: OAK-3794 > URL: https://issues.apache.org/jira/browse/OAK-3794 > Project: Jackrabbit Oak > Issue Type: Bug > Components: tarmk-standby >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.4, 1.3.13, 1.2.16 > > Attachments: OAK-3794-01.patch, OAK-3794-02.patch > > > The fix for OAK-1828 introduces the possibility of loops in the segment > graph. The Cold Standby client currently traverses the segment graph without > keeping track of the visited segments. This makes the Cold Standby prone to > infinite loops during one transfer operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3794) The Cold Standby should expect loops in the segment graph
[ https://issues.apache.org/jira/browse/OAK-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-3794: - Fix Version/s: 1.2.16 > The Cold Standby should expect loops in the segment graph > - > > Key: OAK-3794 > URL: https://issues.apache.org/jira/browse/OAK-3794 > Project: Jackrabbit Oak > Issue Type: Bug > Components: tarmk-standby >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.4, 1.3.13, 1.2.16 > > Attachments: OAK-3794-01.patch, OAK-3794-02.patch > > > The fix for OAK-1828 introduces the possibility of loops in the segment > graph. The Cold Standby client currently traverses the segment graph without > keeping track of the visited segments. This makes the Cold Standby prone to > infinite loops during one transfer operation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3273) ColdStandby make sync start and end timestamp updates atomic
[ https://issues.apache.org/jira/browse/OAK-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318634#comment-15318634 ] Alex Parvulescu commented on OAK-3273: -- merged to 1.2 with r1747248 > ColdStandby make sync start and end timestamp updates atomic > > > Key: OAK-3273 > URL: https://issues.apache.org/jira/browse/OAK-3273 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: tarmk-standby >Reporter: Valentin Olteanu >Assignee: Alex Parvulescu >Priority: Minor > Fix For: 1.4, 1.3.5, 1.2.16 > > Attachments: OAK-3273.patch > > > OAK-3113 introduced two fields in the ColdStandby MBean: SyncStartTimestamp > and SyncEndTimestamp. This is much more useful than the old > SecondsSinceLastSuccess, yet, there are situations in which it's hard to > interpret them since they are updated independently: > - it's impossible to correlate the start with the end > - in case of fail, the start still reflects the failed cycle > It would be even better if the two would be updated atomically, to reflect > the start and end of the last successful cycle. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3273) ColdStandby make sync start and end timestamp updates atomic
[ https://issues.apache.org/jira/browse/OAK-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-3273: - Fix Version/s: 1.2.16 > ColdStandby make sync start and end timestamp updates atomic > > > Key: OAK-3273 > URL: https://issues.apache.org/jira/browse/OAK-3273 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: tarmk-standby >Reporter: Valentin Olteanu >Assignee: Alex Parvulescu >Priority: Minor > Fix For: 1.4, 1.3.5, 1.2.16 > > Attachments: OAK-3273.patch > > > OAK-3113 introduced two fields in the ColdStandby MBean: SyncStartTimestamp > and SyncEndTimestamp. This is much more useful than the old > SecondsSinceLastSuccess, yet, there are situations in which it's hard to > interpret them since they are updated independently: > - it's impossible to correlate the start with the end > - in case of fail, the start still reflects the failed cycle > It would be even better if the two would be updated atomically, to reflect > the start and end of the last successful cycle. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3113) ColdStandby should provide sync start and end timestamps
[ https://issues.apache.org/jira/browse/OAK-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318632#comment-15318632 ] Alex Parvulescu commented on OAK-3113: -- merged to 1.2 with r1747247 > ColdStandby should provide sync start and end timestamps > > > Key: OAK-3113 > URL: https://issues.apache.org/jira/browse/OAK-3113 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: tarmk-standby >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu >Priority: Minor > Fix For: 1.4, 1.3.3, 1.2.16 > > > This is for adding more info to the JMX stats that will help with automation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3113) ColdStandby should provide sync start and end timestamps
[ https://issues.apache.org/jira/browse/OAK-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-3113: - Fix Version/s: 1.2.16 > ColdStandby should provide sync start and end timestamps > > > Key: OAK-3113 > URL: https://issues.apache.org/jira/browse/OAK-3113 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: tarmk-standby >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu >Priority: Minor > Fix For: 1.4, 1.3.3, 1.2.16 > > > This is for adding more info to the JMX stats that will help with automation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4439) Fix the errors reported by the Javadoc tool in JDK8
[ https://issues.apache.org/jira/browse/OAK-4439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francesco Mari resolved OAK-4439. - Resolution: Fixed Fix Version/s: 1.5.4 Fixed at r1747246. > Fix the errors reported by the Javadoc tool in JDK8 > --- > > Key: OAK-4439 > URL: https://issues.apache.org/jira/browse/OAK-4439 > Project: Jackrabbit Oak > Issue Type: Bug >Reporter: Francesco Mari >Assignee: Francesco Mari > Fix For: 1.6, 1.5.4 > > > Some Javadoc is not strict enough according to the Javadoc tool shipped in > JDK8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4439) Fix the errors reported by the Javadoc tool in JDK8
Francesco Mari created OAK-4439: --- Summary: Fix the errors reported by the Javadoc tool in JDK8 Key: OAK-4439 URL: https://issues.apache.org/jira/browse/OAK-4439 Project: Jackrabbit Oak Issue Type: Bug Reporter: Francesco Mari Assignee: Francesco Mari Fix For: 1.6 Some Javadoc is not strict enough according to the Javadoc tool shipped in JDK8. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4170) QueryEngine adding invalid property restriction for fulltext query
[
https://issues.apache.org/jira/browse/OAK-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318583#comment-15318583
]
Thomas Mueller commented on OAK-4170:
-
I will remove the "not null" condition ("2" on your list). But first I fixed
the actual bug in http://svn.apache.org/r1747245, in case we want to re-add it
later on for some reason.
> QueryEngine adding invalid property restriction for fulltext query
> --
>
> Key: OAK-4170
> URL: https://issues.apache.org/jira/browse/OAK-4170
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: query
>Reporter: Chetan Mehrotra
>Assignee: Thomas Mueller
>Priority: Minor
> Fix For: 1.6
>
> Attachments: OAK-4170-v1.patch
>
>
> QueryEngine inserts a property restriction of "is not null" for any property
> used in fulltext constraint. For e.g. for query
> {noformat}
> select * from [nt:unstructured] where
> CONTAINS([jcr:content/metadata/comment], 'december')
> {noformat}
> A property restriction would be added for {{jcr:content/metadata/comment}}.
> However currently due to bug in {{FulltextSearchImpl}} [1] the property name
> generated is {{comment/jcr:content/metadata}}.
> {code}
> @Override
> public void restrict(FilterImpl f) {
> if (propertyName != null) {
> if (f.getSelector().equals(selector)) {
> String p = propertyName;
> if (relativePath != null) {
> p = PathUtils.concat(p, relativePath);
> }
> p = normalizePropertyName(p);
> restrictPropertyOnFilter(p, f);
> }
> }
>
> f.restrictFulltextCondition(fullTextSearchExpression.currentValue().getValue(Type.STRING));
> }
> {code}
> This happens because {{relativePath}} is passed as second param to
> {{PathUtils.concat}}. It should be first param
> [1]
> https://github.com/apache/jackrabbit-oak/blob/1.4/oak-core/src/main/java/org/apache/jackrabbit/oak/query/ast/FullTextSearchImpl.java#L275-L286
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4438) Segments created by an unsuccessful compaction run should get cleaned
[ https://issues.apache.org/jira/browse/OAK-4438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig resolved OAK-4438. Resolution: Fixed Fixed at http://svn.apache.org/viewvc?rev=1747240&view=rev > Segments created by an unsuccessful compaction run should get cleaned > - > > Key: OAK-4438 > URL: https://issues.apache.org/jira/browse/OAK-4438 > Project: Jackrabbit Oak > Issue Type: Bug > Components: segment-tar >Reporter: Michael Dürig >Assignee: Michael Dürig > Labels: compaction, gc > Fix For: 1.6 > > > Cleaning of segment created by an unsuccessful compaction run currently only > works if forced compaction is enabled. Otherwise those segments will only get > cleaned in a much later cleanup cycle. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4438) Segments created by an unsuccessful compaction run should get cleaned
Michael Dürig created OAK-4438: -- Summary: Segments created by an unsuccessful compaction run should get cleaned Key: OAK-4438 URL: https://issues.apache.org/jira/browse/OAK-4438 Project: Jackrabbit Oak Issue Type: Bug Components: segment-tar Reporter: Michael Dürig Assignee: Michael Dürig Fix For: 1.6 Cleaning of segment created by an unsuccessful compaction run currently only works if forced compaction is enabled. Otherwise those segments will only get cleaned in a much later cleanup cycle. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4437) Backport OAK-4387 (XPath: querying for nodes named "text"...) to 1.2 and 1.4
[ https://issues.apache.org/jira/browse/OAK-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller resolved OAK-4437. - Resolution: Fixed > Backport OAK-4387 (XPath: querying for nodes named "text"...) to 1.2 and 1.4 > > > Key: OAK-4437 > URL: https://issues.apache.org/jira/browse/OAK-4437 > Project: Jackrabbit Oak > Issue Type: Bug > Components: query >Affects Versions: 1.2, 1.4 >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.2.16, 1.4.4 > > > XPath: querying for nodes named "text", "element", and "rep:excerpt" fails -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4437) Backport OAK-4387 (XPath: querying for nodes named "text"...) to 1.2 and 1.4
[ https://issues.apache.org/jira/browse/OAK-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318456#comment-15318456 ] Thomas Mueller commented on OAK-4437: - http://svn.apache.org/r1747224 (1.2 branch) http://svn.apache.org/r1747228 (1.4 branch) > Backport OAK-4387 (XPath: querying for nodes named "text"...) to 1.2 and 1.4 > > > Key: OAK-4437 > URL: https://issues.apache.org/jira/browse/OAK-4437 > Project: Jackrabbit Oak > Issue Type: Bug > Components: query >Affects Versions: 1.2, 1.4 >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.2.16, 1.4.4 > > > XPath: querying for nodes named "text", "element", and "rep:excerpt" fails -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4437) Backport OAK-4387 (XPath: querying for nodes named "text"...) to 1.2 and 1.4
[ https://issues.apache.org/jira/browse/OAK-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-4437: Summary: Backport OAK-4387 (XPath: querying for nodes named "text"...) to 1.2 and 1.4 (was: Backport OAK-3991 (XPath: querying for nodes named "text"...) to 1.2 and 1.4) > Backport OAK-4387 (XPath: querying for nodes named "text"...) to 1.2 and 1.4 > > > Key: OAK-4437 > URL: https://issues.apache.org/jira/browse/OAK-4437 > Project: Jackrabbit Oak > Issue Type: Bug > Components: query >Affects Versions: 1.2, 1.4 >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.2.16, 1.4.4 > > > XPath: querying for nodes named "text", "element", and "rep:excerpt" fails -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info
[
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318366#comment-15318366
]
Michael Dürig commented on OAK-4420:
Cannot comment on the details re. the sidegrade itself. But it looks like what
I was proposing in my earlier comment. Is there a test case actually asserting
that checkpoints get de-duplicated?
Regarding the super root accessors, are they really needed? I would prefer if
we would operate directly on the {{FileStore}} / {{Revisions}} APIs.
> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
> Attachments: OAK-4420-naive.patch, OAK-4420.patch
>
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take
> care of moving the checkpoint data and meta. This will save a very expensive
> full-reindex.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4373) Refactor SegmentTracker
[
https://issues.apache.org/jira/browse/OAK-4373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-4373.
Resolution: Fixed
The bulk of the work is done with my latest changes. Will report dedicated
issues if needed.
> Refactor SegmentTracker
> ---
>
> Key: OAK-4373
> URL: https://issues.apache.org/jira/browse/OAK-4373
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: technical_debt
> Fix For: 1.6
>
>
> The {{SegmentTracker}} class has become the dumping ground for everything
> that wouldn't fit else where. In a personal discussion with [~frm], we
> figured that this class might be a good starting point refactoring
> {{segment-tar}} towards better encapsulation.
> The aim would be to return {{SegmentTracker}} to its initial purpose (i.e.
> tracking segments) and move all unrelated concerns elsewhere.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4428) Optimize RevisionVector methods
[
https://issues.apache.org/jira/browse/OAK-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-4428.
--
Resolution: Fixed
Fix Version/s: 1.5.4
Thanks Thomas for the review
Fixed with
* trunk - 1747197
> Optimize RevisionVector methods
> ---
>
> Key: OAK-4428
> URL: https://issues.apache.org/jira/browse/OAK-4428
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Labels: performance
> Fix For: 1.6, 1.5.4
>
> Attachments: OAK-4428-v1.patch, OAK-4428-v2.patch, hot-methods-1.png
>
>
> {{RevisionVector}} is used in very critical paths and we should look into
> optimzing some of its critical method
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4431) Index path property should be considered optional for copy on read logic
[
https://issues.apache.org/jira/browse/OAK-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-4431.
--
Resolution: Fixed
Fix Version/s: 1.5.4
Done
* trunk - 1747198,1747200
> Index path property should be considered optional for copy on read logic
>
>
> Key: OAK-4431
> URL: https://issues.apache.org/jira/browse/OAK-4431
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Fix For: 1.6, 1.5.4
>
>
> As part of changes done for OAK-4347 logic assumes that indexPath is always
> non null. This works fine for fresh setup where the indexPath would have been
> set by the initial indexing. However for upgraded setup this assumption would
> break as it might happen that index does not get updated with new approach
> and before that a read is performed.
> Currently with updated code on upgraded setup following exception is seen
> {noformat}
> Caused by: javax.security.auth.login.LoginException:
> java.lang.NullPointerException: Index path property [:indexPath] not found
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:236)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexDefinition.getIndexPathFromConfig(IndexDefinition.java:664)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier.getSharedWorkingSet(IndexCopier.java:242)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier.wrapForRead(IndexCopier.java:140)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexNode.open(IndexNode.java:53)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.findIndexNode(IndexTracker.java:179)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.IndexTracker.acquireIndexNode(IndexTracker.java:154)
> at
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex.getPlans(LucenePropertyIndex.java:250)
> {noformat}
> For this specific flow the indexPath can be passed in and not looked up from
> IndexDefinition
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4427) NodeDocument.fromString should also seal the returned document
[ https://issues.apache.org/jira/browse/OAK-4427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-4427. -- Resolution: Fixed Fix Version/s: 1.5.4 Done * trunk - 1747199 > NodeDocument.fromString should also seal the returned document > -- > > Key: OAK-4427 > URL: https://issues.apache.org/jira/browse/OAK-4427 > Project: Jackrabbit Oak > Issue Type: Task > Components: documentmk >Reporter: Chetan Mehrotra >Assignee: Chetan Mehrotra >Priority: Minor > Fix For: 1.6, 1.5.4 > > > NodeDocument which gets created from JSON string (from persisted cache) > should also be sealed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-4102) Break cyclic dependency of FileStore and SegmentTracker
[
https://issues.apache.org/jira/browse/OAK-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari resolved OAK-4102.
-
Resolution: Fixed
Fix Version/s: 1.5.4
Fixed at r1747196. The committed version includes above suggestions by
[~mduerig].
> Break cyclic dependency of FileStore and SegmentTracker
> ---
>
> Key: OAK-4102
> URL: https://issues.apache.org/jira/browse/OAK-4102
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: technical_debt
> Fix For: 1.6, 1.5.4
>
> Attachments: OAK-4102-01.patch, OAK-4102-02.patch, OAK-4102-03.patch
>
>
> {{SegmentTracker}} and {{FileStore}} are mutually dependent on each other.
> This is problematic and makes initialising instances of these classes
> difficult: the {{FileStore}} constructor e.g. passes a not fully initialised
> instance to the {{SegmentTracker}}, which in turn writes an initial node
> state to the store. Notably using the not fully initialised {{FileStore}}
> instance!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info
[
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318304#comment-15318304
]
Tomek Rękawek edited comment on OAK-4420 at 6/7/16 11:19 AM:
-
I prepared a proper version of patch: [^OAK-4420.patch]. The main part is the
{{RepositorySidegrade#copyCheckpoints()}}. The NodeStateCopier used there to
merge recent changes is intelligent enough to copy just the diff. Also, I've
modified both SegmentNodeStore implementations to allow setting a new super
root.
[~alex.parvulescu], [~mduerig], could you take a look on it? If you're ok with
the SegmentNodeStore changes I'll merge it,
was (Author: tomek.rekawek):
I prepared a proper version of patch: [^OAK-4420.patch]. The main part is the
{{RepositorySidegrade#copyCheckpoints()}}. The NodeStateCopier used there to
merge recent changes is intelligent enough to copy just the diff. Also, I've
modified both SegmentNodeStore implementations to allow setting a new super
root.
[~alex.parvulescu], [~mduerig], could you take a look on it?
> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
> Attachments: OAK-4420-naive.patch, OAK-4420.patch
>
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take
> care of moving the checkpoint data and meta. This will save a very expensive
> full-reindex.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info
[
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318304#comment-15318304
]
Tomek Rękawek commented on OAK-4420:
I prepared a proper version of patch: [^OAK-4420.patch]. The main part is the
{{RepositorySidegrade#copyCheckpoints()}}. The NodeStateCopier used there to
merge recent changes is intelligent enough to copy just the diff. Also, I've
modified both SegmentNodeStore implementations to allow setting a new super
root.
[~alex.parvulescu], [~mduerig], could you take a look on it?
> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
> Attachments: OAK-4420-naive.patch, OAK-4420.patch
>
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take
> care of moving the checkpoint data and meta. This will save a very expensive
> full-reindex.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info
[
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-4420:
---
Attachment: OAK-4420.patch
> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
> Attachments: OAK-4420-naive.patch, OAK-4420.patch
>
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take
> care of moving the checkpoint data and meta. This will save a very expensive
> full-reindex.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4102) Break cyclic dependency of FileStore and SegmentTracker
[
https://issues.apache.org/jira/browse/OAK-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318278#comment-15318278
]
Michael Dürig commented on OAK-4102:
Good stuff, I like it! Looking at the usages of the tracker we are with this
patch only a small step away from removing those {{getTracker}} methods from
the individual {{SegmentStore}} implementations. So let's apply it!
A few minor points:
* I don't particular like the name {{SegmentIdMaker}} and would prefer
{{SegmentIdFactory}}. Also the private instance variable should probably be
renamed to {{idFactory}}.
* While we are at it we should clarify the naming and semantics of the various
{{(get|new)(Bulk|Data)?SegmentId}} methods in the {{SegmentStore}}
implementations, the {{SegmentTracker}} and the {{SegmentIdMaker}}. Some of
them are "real" factories others getters with a factory fall back. The naming
should be consistent with that.
* I think we should add nullability annotations where applicable.
> Break cyclic dependency of FileStore and SegmentTracker
> ---
>
> Key: OAK-4102
> URL: https://issues.apache.org/jira/browse/OAK-4102
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: technical_debt
> Fix For: 1.6
>
> Attachments: OAK-4102-01.patch, OAK-4102-02.patch, OAK-4102-03.patch
>
>
> {{SegmentTracker}} and {{FileStore}} are mutually dependent on each other.
> This is problematic and makes initialising instances of these classes
> difficult: the {{FileStore}} constructor e.g. passes a not fully initialised
> instance to the {{SegmentTracker}}, which in turn writes an initial node
> state to the store. Notably using the not fully initialised {{FileStore}}
> instance!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4437) Backport OAK-3991 (XPath: querying for nodes named "text"...) to 1.2 and 1.4
[ https://issues.apache.org/jira/browse/OAK-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-4437: Fix Version/s: 1.4.4 1.2.16 > Backport OAK-3991 (XPath: querying for nodes named "text"...) to 1.2 and 1.4 > > > Key: OAK-4437 > URL: https://issues.apache.org/jira/browse/OAK-4437 > Project: Jackrabbit Oak > Issue Type: Bug > Components: query >Affects Versions: 1.2, 1.4 >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.2.16, 1.4.4 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4437) Backport OAK-3991 (XPath: querying for nodes named "text"...) to 1.2 and 1.4
[ https://issues.apache.org/jira/browse/OAK-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-4437: Component/s: query > Backport OAK-3991 (XPath: querying for nodes named "text"...) to 1.2 and 1.4 > > > Key: OAK-4437 > URL: https://issues.apache.org/jira/browse/OAK-4437 > Project: Jackrabbit Oak > Issue Type: Bug > Components: query >Affects Versions: 1.2, 1.4 >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.2.16, 1.4.4 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4437) Backport OAK-3991 (XPath: querying for nodes named "text"...) to 1.2 and 1.4
[ https://issues.apache.org/jira/browse/OAK-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-4437: Description: XPath: querying for nodes named "text", "element", and "rep:excerpt" fails > Backport OAK-3991 (XPath: querying for nodes named "text"...) to 1.2 and 1.4 > > > Key: OAK-4437 > URL: https://issues.apache.org/jira/browse/OAK-4437 > Project: Jackrabbit Oak > Issue Type: Bug > Components: query >Affects Versions: 1.2, 1.4 >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.2.16, 1.4.4 > > > XPath: querying for nodes named "text", "element", and "rep:excerpt" fails -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4437) Backport OAK-3991 (XPath: querying for nodes named "text"...) to 1.2 and 1.4
[ https://issues.apache.org/jira/browse/OAK-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-4437: Affects Version/s: 1.2 1.4 > Backport OAK-3991 (XPath: querying for nodes named "text"...) to 1.2 and 1.4 > > > Key: OAK-4437 > URL: https://issues.apache.org/jira/browse/OAK-4437 > Project: Jackrabbit Oak > Issue Type: Bug > Components: query >Affects Versions: 1.2, 1.4 >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.2.16, 1.4.4 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-4437) Backport OAK-3991 (XPath: querying for nodes named "text"...) to 1.2 and 1.4
Thomas Mueller created OAK-4437: --- Summary: Backport OAK-3991 (XPath: querying for nodes named "text"...) to 1.2 and 1.4 Key: OAK-4437 URL: https://issues.apache.org/jira/browse/OAK-4437 Project: Jackrabbit Oak Issue Type: Bug Reporter: Thomas Mueller Assignee: Thomas Mueller -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4102) Break cyclic dependency of FileStore and SegmentTracker
[
https://issues.apache.org/jira/browse/OAK-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-4102:
Attachment: OAK-4102-03.patch
The previous patch is broken. [^OAK-4102-03.patch] includes the missing files.
> Break cyclic dependency of FileStore and SegmentTracker
> ---
>
> Key: OAK-4102
> URL: https://issues.apache.org/jira/browse/OAK-4102
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: technical_debt
> Fix For: 1.6
>
> Attachments: OAK-4102-01.patch, OAK-4102-02.patch, OAK-4102-03.patch
>
>
> {{SegmentTracker}} and {{FileStore}} are mutually dependent on each other.
> This is problematic and makes initialising instances of these classes
> difficult: the {{FileStore}} constructor e.g. passes a not fully initialised
> instance to the {{SegmentTracker}}, which in turn writes an initial node
> state to the store. Notably using the not fully initialised {{FileStore}}
> instance!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4422) support cluster for FileBlobStore
[ https://issues.apache.org/jira/browse/OAK-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318209#comment-15318209 ] Marco Piovesana commented on OAK-4422: -- thanks Davide, yes this helps. I'm quite new to Oak and I was still a bit confused between Datastore and Nodestore. I do have one more question though: how do i specify the dedicated FileBlobStore vs the shared Datastore? I read the oak guide and i tried to find an example on internet but i did not find anything. thanks, Marco. > support cluster for FileBlobStore > - > > Key: OAK-4422 > URL: https://issues.apache.org/jira/browse/OAK-4422 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: blob >Affects Versions: 1.4.3 >Reporter: Marco Piovesana > > I'm using Oak in a system where the user can store arbitrary large binary > files and because of that I thought the best option was to use the > FileBlobStore as storage subsystem. > Now I need to port this solution on a cluster environment, but i saw that > clustering is supported only for Mongo and RDBMS storage systems. Is there > any plan to suppor it also for the Blob storage? There's a better option? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-4428) Optimize RevisionVector methods
[
https://issues.apache.org/jira/browse/OAK-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Mueller updated OAK-4428:
Attachment: OAK-4428-v2.patch
Updated patch (some formatting fixes to conform checkstyle)
> Optimize RevisionVector methods
> ---
>
> Key: OAK-4428
> URL: https://issues.apache.org/jira/browse/OAK-4428
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
>Priority: Minor
> Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-4428-v1.patch, OAK-4428-v2.patch, hot-methods-1.png
>
>
> {{RevisionVector}} is used in very critical paths and we should look into
> optimzing some of its critical method
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4421) Optimize Revison fromString and toString implementation
[
https://issues.apache.org/jira/browse/OAK-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Mueller updated OAK-4421:
Attachment: OAK-4421-v3.patch
Updated patch (incompatibilities fixed)
> Optimize Revison fromString and toString implementation
> ---
>
> Key: OAK-4421
> URL: https://issues.apache.org/jira/browse/OAK-4421
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-4421-v1.patch, OAK-4421-v2.patch, OAK-4421-v3.patch,
> hot-methods.png, revision-fromstring-caller.png
>
>
> Current implementation of Revision {{fromString}} and {{toString}} make use
> of std JDK API to perform string manipulation. While running some performance
> test it was seen that these 2 methods are called quite frequently and that
> adds up to some decent times. Further they also generate quite a bit of short
> lived objects.
> !hot-methods.png!
> It would be worthwhile to perform a micro benchmark of these method and
> optimize them further such that they perform better and also generate less
> garbage. The micro optimized code would be bit more complex but if
> performance numbers are better we can look into changing the current
> implementation
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4102) Break cyclic dependency of FileStore and SegmentTracker
[
https://issues.apache.org/jira/browse/OAK-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-4102:
Attachment: OAK-4102-02.patch
[^OAK-4102-02.patch] contains the same changes as the previous patch rebased to
the latest trunk. In addition, I added Javadoc on {{SegmentIdMaker}} and
{{SegmentIdTable}}.
> Break cyclic dependency of FileStore and SegmentTracker
> ---
>
> Key: OAK-4102
> URL: https://issues.apache.org/jira/browse/OAK-4102
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
> Labels: technical_debt
> Fix For: 1.6
>
> Attachments: OAK-4102-01.patch, OAK-4102-02.patch
>
>
> {{SegmentTracker}} and {{FileStore}} are mutually dependent on each other.
> This is problematic and makes initialising instances of these classes
> difficult: the {{FileStore}} constructor e.g. passes a not fully initialised
> instance to the {{SegmentTracker}}, which in turn writes an initial node
> state to the store. Notably using the not fully initialised {{FileStore}}
> instance!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info
[
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318131#comment-15318131
]
Tomek Rękawek commented on OAK-4420:
Great idea, thanks! I'll take care of this.
> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
> Attachments: OAK-4420-naive.patch
>
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take
> care of moving the checkpoint data and meta. This will save a very expensive
> full-reindex.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4422) support cluster for FileBlobStore
[
https://issues.apache.org/jira/browse/OAK-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318119#comment-15318119
]
Davide Giannella commented on OAK-4422:
---
I think there's a bit of confusion in terminology. Or maybe it's me
not understanding.
When you speak of FileBlobStore I associate it to the Datastore which
is "different" from the {{NodeStore}}. The NodeStore is the part you
think about clustering using a DocumentNodeStore implementation.
AFAIK you can configure any of our DocumentNS implementation (Mongo
and RDB) with a FileBlobStore.
When it comes to clustering you can then decide to dedicate a
FileBlobStore for each individual cluster node, having therefore
multiple copies of the same binary, or opt for a shared Datastore to
be used by all the cluster nodes.
Don't remember if we have any utilities for moving from Segment to
Document.
Would this solve your situation?
> support cluster for FileBlobStore
> -
>
> Key: OAK-4422
> URL: https://issues.apache.org/jira/browse/OAK-4422
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: blob
>Affects Versions: 1.4.3
>Reporter: Marco Piovesana
>
> I'm using Oak in a system where the user can store arbitrary large binary
> files and because of that I thought the best option was to use the
> FileBlobStore as storage subsystem.
> Now I need to port this solution on a cluster environment, but i saw that
> clustering is supported only for Mongo and RDBMS storage systems. Is there
> any plan to suppor it also for the Blob storage? There's a better option?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info
[
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4420:
---
Fix Version/s: 1.6
> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
> Attachments: OAK-4420-naive.patch
>
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take
> care of moving the checkpoint data and meta. This will save a very expensive
> full-reindex.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4436) HeavyWriteIT sporadically fails
Michael Dürig created OAK-4436:
--
Summary: HeavyWriteIT sporadically fails
Key: OAK-4436
URL: https://issues.apache.org/jira/browse/OAK-4436
Project: Jackrabbit Oak
Issue Type: Bug
Components: segment-tar
Reporter: Michael Dürig
Priority: Critical
Fix For: 1.6
I've seen {{HeavyWriteIT}} fail sporadically on my local checkout.
{noformat}
3d13e2927fc0d75454a692ef5c8703880dc2ea0d
org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment
31b75992-aaf7-4f2b-a5de-b5a268c1fdb3 not found
at
org.apache.jackrabbit.oak.segment.file.FileStore$14.call(FileStore.java:1377)
at
org.apache.jackrabbit.oak.segment.file.FileStore$14.call(FileStore.java:1317)
at
org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1011)
at
org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:972)
at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:283)
at
org.apache.jackrabbit.oak.segment.SegmentCache.geSegment(SegmentCache.java:80)
at
org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1317)
at
org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:111)
at
org.apache.jackrabbit.oak.segment.RecordId.getSegment(RecordId.java:94)
at
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.isOldGeneration(SegmentWriter.java:1010)
at
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNodeUncached(SegmentWriter.java:906)
at
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(SegmentWriter.java:885)
at
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.access$700(SegmentWriter.java:319)
at
org.apache.jackrabbit.oak.segment.SegmentWriter$8.execute(SegmentWriter.java:277)
at
org.apache.jackrabbit.oak.segment.SegmentBufferWriterPool.execute(SegmentBufferWriterPool.java:110)
at
org.apache.jackrabbit.oak.segment.SegmentWriter.writeNode(SegmentWriter.java:274)
at
org.apache.jackrabbit.oak.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:111)
at
org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.(SegmentNodeStore.java:516)
at
org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:284)
at
org.apache.jackrabbit.oak.segment.HeavyWriteIT.heavyWrite(HeavyWriteIT.java:91)
{noformat}
I suspect this is a problem with {{isOldGeneration}} itself not being prepared
for the old segment actually being gone.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4435) checkpointDeduplicationTest sometimes fails on Jenkins
[
https://issues.apache.org/jira/browse/OAK-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4435:
---
Description:
{{CompactionAndCleanupIT.checkpointDeduplication}} irregularly
[fails|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/938/jdk=latest1.7,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/console]
on Jenkins.
This might point to an issue with the de-duplication caches, which are crucial
in getting the checkpoints de-duplicated.
{code}
checkpointDeduplicationTest(org.apache.jackrabbit.oak.segment.CompactionAndCleanupIT)
Time elapsed: 0.15 sec <<< FAILURE!
org.junit.ComparisonFailure:
expected:<[7211975a-04ce-45ff-aff5-16795ec2cc72]:261932> but
was:<[11083c4b-9b2e-4d17-a8c0-8f6b1f2a3173]:261932>
at org.junit.Assert.assertEquals(Assert.java:115)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.jackrabbit.oak.segment.CompactionAndCleanupIT.checkpointDeduplicationTest(CompactionAndCleanupIT.java:899)
{code}
was:
{{CompactionAndCleanupIT.checkpointDeduplication}} irregularly
[fails|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/938/jdk=latest1.7,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/console]
on Jenkins.
This might point to an issue with the de-duplication caches, which are crucial
in getting the checkpoints de-duplicated.
> checkpointDeduplicationTest sometimes fails on Jenkins
> --
>
> Key: OAK-4435
> URL: https://issues.apache.org/jira/browse/OAK-4435
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Priority: Critical
> Labels: compaction, gc, test
> Fix For: 1.6
>
>
> {{CompactionAndCleanupIT.checkpointDeduplication}} irregularly
> [fails|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/938/jdk=latest1.7,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/console]
> on Jenkins.
> This might point to an issue with the de-duplication caches, which are
> crucial in getting the checkpoints de-duplicated.
> {code}
> checkpointDeduplicationTest(org.apache.jackrabbit.oak.segment.CompactionAndCleanupIT)
> Time elapsed: 0.15 sec <<< FAILURE!
> org.junit.ComparisonFailure:
> expected:<[7211975a-04ce-45ff-aff5-16795ec2cc72]:261932> but
> was:<[11083c4b-9b2e-4d17-a8c0-8f6b1f2a3173]:261932>
> at org.junit.Assert.assertEquals(Assert.java:115)
> at org.junit.Assert.assertEquals(Assert.java:144)
> at
> org.apache.jackrabbit.oak.segment.CompactionAndCleanupIT.checkpointDeduplicationTest(CompactionAndCleanupIT.java:899)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4435) checkpointDeduplicationTest sometimes fails on Jenkins
[
https://issues.apache.org/jira/browse/OAK-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4435:
---
Priority: Critical (was: Major)
> checkpointDeduplicationTest sometimes fails on Jenkins
> --
>
> Key: OAK-4435
> URL: https://issues.apache.org/jira/browse/OAK-4435
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
>Priority: Critical
> Labels: compaction, gc, test
> Fix For: 1.6
>
>
> {{CompactionAndCleanupIT.checkpointDeduplication}} irregularly
> [fails|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/938/jdk=latest1.7,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/console]
> on Jenkins.
> This might point to an issue with the de-duplication caches, which are
> crucial in getting the checkpoints de-duplicated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4435) checkpointDeduplicationTest sometimes fails on Jenkins
[
https://issues.apache.org/jira/browse/OAK-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-4435:
---
Fix Version/s: 1.6
> checkpointDeduplicationTest sometimes fails on Jenkins
> --
>
> Key: OAK-4435
> URL: https://issues.apache.org/jira/browse/OAK-4435
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Michael Dürig
> Labels: compaction, gc, test
> Fix For: 1.6
>
>
> {{CompactionAndCleanupIT.checkpointDeduplication}} irregularly
> [fails|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/938/jdk=latest1.7,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/console]
> on Jenkins.
> This might point to an issue with the de-duplication caches, which are
> crucial in getting the checkpoints de-duplicated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4421) Optimize Revison fromString and toString implementation
[
https://issues.apache.org/jira/browse/OAK-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318103#comment-15318103
]
Thomas Mueller commented on OAK-4421:
-
> with this patch RevisionTest.fromStringToString is failing for me
Sorry, there was a typo in Revision.fromString, "clusterId +=" instead of
"clusterId =". I will upload a new patch.
> I'm wondering why these methods are a hotspot.
In this test case, it looks like the persistent cache calls does a lot of
de-serialization and serialization. Right now, the string / JSON representation
is used, which seems to be the bottleneck here. Sure, binary formatting would
be faster; I didn't do that because I thought it's much harder to debug and
maintain. So before doing that we should analyze if the number of calls can be
reduced. Good to know this could be related to OAK-4358.
> Optimize Revison fromString and toString implementation
> ---
>
> Key: OAK-4421
> URL: https://issues.apache.org/jira/browse/OAK-4421
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: documentmk
>Reporter: Chetan Mehrotra
>Assignee: Chetan Mehrotra
> Labels: performance
> Fix For: 1.6
>
> Attachments: OAK-4421-v1.patch, OAK-4421-v2.patch, hot-methods.png,
> revision-fromstring-caller.png
>
>
> Current implementation of Revision {{fromString}} and {{toString}} make use
> of std JDK API to perform string manipulation. While running some performance
> test it was seen that these 2 methods are called quite frequently and that
> adds up to some decent times. Further they also generate quite a bit of short
> lived objects.
> !hot-methods.png!
> It would be worthwhile to perform a micro benchmark of these method and
> optimize them further such that they perform better and also generate less
> garbage. The micro optimized code would be bit more complex but if
> performance numbers are better we can look into changing the current
> implementation
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4435) checkpointDeduplicationTest sometimes fails on Jenkins
Michael Dürig created OAK-4435:
--
Summary: checkpointDeduplicationTest sometimes fails on Jenkins
Key: OAK-4435
URL: https://issues.apache.org/jira/browse/OAK-4435
Project: Jackrabbit Oak
Issue Type: Bug
Components: segment-tar
Reporter: Michael Dürig
{{CompactionAndCleanupIT.checkpointDeduplication}} irregularly
[fails|https://builds.apache.org/job/Apache%20Jackrabbit%20Oak%20matrix/938/jdk=latest1.7,label=Ubuntu,nsfixtures=SEGMENT_MK,profile=integrationTesting/console]
on Jenkins.
This might point to an issue with the de-duplication caches, which are crucial
in getting the checkpoints de-duplicated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4277) Finalise de-duplication caches
[
https://issues.apache.org/jira/browse/OAK-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318086#comment-15318086
]
Michael Dürig commented on OAK-4277:
Some initial work:
* http://svn.apache.org/viewvc?rev=1747158&view=rev decouples the
{{WriteCacheManager}} from the {{SegmentWriter}}. Instead of being instantiated
by the latter it is now passed into it via a constructor argument.
* http://svn.apache.org/viewvc?rev=1747159&view=rev makes {{FileStoreBuilder}}
a top level class. This builder has a lot of its own functionality by now and
will probably accumulate more once we need to pass cache configuration through
it.
* http://svn.apache.org/viewvc?rev=1747160&view=rev: Invalidating the write
caches in {{WriteCacheManager}} is now down via an (extension of) the
{{GCMonitor}} instead of an explicit method call to the cache. This further
reduces coupling between {{SegmentWriter}}, {{FileStore}} and the caches.
> Finalise de-duplication caches
> --
>
> Key: OAK-4277
> URL: https://issues.apache.org/jira/browse/OAK-4277
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: caching, compaction, gc, monitoring
> Fix For: 1.6
>
>
> OAK-3348 "promoted" the record cache to a de-duplication cache, which is
> heavily relied upon during compaction. Now also node states go through this
> cache, which can seen as one concern of the former compaction map (the other
> being equality).
> The current implementation of these caches is quite simple and served its
> purpose for a POC for getting rid of the "back references" (OAK-3348). Before
> we are ready for a release we need to finalise a couple of things though:
> * Implement cache monitoring and management
> * Make cache parameters now hard coded configurable
> * Implement proper UTs
> * Add proper Javadoc
> * Fine tune eviction logic and move it into the caches themselves (instead of
> relying on the client to evict items pro-actively)
> * Fine tune caching strategies: For the node state cache the cost of the item
> is determined just by its position in the tree. We might want to take further
> things into account (e.g. number of child nodes). Also we might want to
> implement pinning so e.g. checkpoints would never be evicted.
> * Finally we need to decide who should own this cache. It currently lives
> with the {{SegmentWriter}}. However this is IMO not the correct location as
> during compaction there is dedicated segment writer whose cache need to be
> shared with the primary's segment writer upon successful completion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4283) Align GCMonitor API with implementation
[
https://issues.apache.org/jira/browse/OAK-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318077#comment-15318077
]
Michael Dürig commented on OAK-4283:
At http://svn.apache.org/viewvc?rev=1747157&view=rev I did some initial
refactoring on duplicated functionality: the {{LoggingGCMonitor}} is now a top
level class. Delegation has been removed as it should be used in conjunction
with {{DelegatingGCMonitor}} if delegating is necessary.
> Align GCMonitor API with implementation
>
>
> Key: OAK-4283
> URL: https://issues.apache.org/jira/browse/OAK-4283
> Project: Jackrabbit Oak
> Issue Type: Task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: api-change, compaction, gc
> Fix For: 1.6
>
>
> The argument taken by {{GCMonitor.compacted}} related to parameters of the
> compaction map. The latter has gone with OAK-3348. We need to come up with a
> way to adjust this API accordingly. Also it might make sense to broaden the
> scope of {{GCMonitor}} from its initial intent (logging) to a more general
> one as this is how it is already used e.g. by the {{RefreshOnGC}}
> implementation and for OAK-4096.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (OAK-4430) DataStoreBlobStore#getAllChunkIds fetches DataRecord when not needed
[
https://issues.apache.org/jira/browse/OAK-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain updated OAK-4430:
---
Comment: was deleted
(was: The method {{DataStoreBlobStore#getAllChunkIds}} also used the DataRecord
fetched to encode the length in the id. Considering that this method has only
one consumer i.e. the {{MarkSweepGarbageCollector}}, we could alter this method
itself to not encode the blob ids with the length and clearly specify in the
javadocs. Alternately, we could add an overloaded method that returns all raw
blob ids.
Either way this would require a method which the gc class can use to get a raw
id given a length encoded id which the "node store referenced blobs"
collection phase would return.
[~chetanm] wdyt?)
> DataStoreBlobStore#getAllChunkIds fetches DataRecord when not needed
>
>
> Key: OAK-4430
> URL: https://issues.apache.org/jira/browse/OAK-4430
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob
>Reporter: Amit Jain
>Assignee: Amit Jain
> Labels: candidate_oak_1_2, candidate_oak_1_4
> Fix For: 1.6
>
>
> DataStoreBlobStore#getAllChunkIds loads the DataRecord for checking that the
> lastModifiedTime criteria is satisfied against the given
> {{maxLastModifiedTime}}.
> When the {{maxLastModifiedTime}} has a value 0 it effectively means ignore
> any last modified time check (and which is the only usage currently from
> MarkSweepGarbageCollector). This should ignore fetching the DataRecords as
> this can be very expensive for e.g on calls to S3 with millions of blobs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4430) DataStoreBlobStore#getAllChunkIds fetches DataRecord when not needed
[
https://issues.apache.org/jira/browse/OAK-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318072#comment-15318072
]
Amit Jain commented on OAK-4430:
Just for the record had an offline chat with [~chetanm] yesterday and he
suggested that since, S3 already returns the size object when doing the listing
it's better we create a new overloaded method which return the DataRecord. This
will help save the additional call to S3 and also not leak out abstraction
related to length encoding in the blob ids.
> DataStoreBlobStore#getAllChunkIds fetches DataRecord when not needed
>
>
> Key: OAK-4430
> URL: https://issues.apache.org/jira/browse/OAK-4430
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob
>Reporter: Amit Jain
>Assignee: Amit Jain
> Labels: candidate_oak_1_2, candidate_oak_1_4
> Fix For: 1.6
>
>
> DataStoreBlobStore#getAllChunkIds loads the DataRecord for checking that the
> lastModifiedTime criteria is satisfied against the given
> {{maxLastModifiedTime}}.
> When the {{maxLastModifiedTime}} has a value 0 it effectively means ignore
> any last modified time check (and which is the only usage currently from
> MarkSweepGarbageCollector). This should ignore fetching the DataRecords as
> this can be very expensive for e.g on calls to S3 with millions of blobs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-4430) DataStoreBlobStore#getAllChunkIds fetches DataRecord when not needed
[
https://issues.apache.org/jira/browse/OAK-4430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain updated OAK-4430:
---
Labels: candidate_oak_1_2 candidate_oak_1_4 (was: candidate_oak_1_0
candidate_oak_1_2 candidate_oak_1_4)
> DataStoreBlobStore#getAllChunkIds fetches DataRecord when not needed
>
>
> Key: OAK-4430
> URL: https://issues.apache.org/jira/browse/OAK-4430
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: blob
>Reporter: Amit Jain
>Assignee: Amit Jain
> Labels: candidate_oak_1_2, candidate_oak_1_4
> Fix For: 1.6
>
>
> DataStoreBlobStore#getAllChunkIds loads the DataRecord for checking that the
> lastModifiedTime criteria is satisfied against the given
> {{maxLastModifiedTime}}.
> When the {{maxLastModifiedTime}} has a value 0 it effectively means ignore
> any last modified time check (and which is the only usage currently from
> MarkSweepGarbageCollector). This should ignore fetching the DataRecords as
> this can be very expensive for e.g on calls to S3 with millions of blobs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-4373) Refactor SegmentTracker
[
https://issues.apache.org/jira/browse/OAK-4373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318071#comment-15318071
]
Michael Dürig commented on OAK-4373:
At http://svn.apache.org/viewvc?rev=1747156&view=rev I refactored the
{{SegmentWriters}} class, which was a static factory for creating
{{SegmentWriter}} instances into a proper builder {{SegmentWriterBuilder}}.
> Refactor SegmentTracker
> ---
>
> Key: OAK-4373
> URL: https://issues.apache.org/jira/browse/OAK-4373
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: technical_debt
> Fix For: 1.6
>
>
> The {{SegmentTracker}} class has become the dumping ground for everything
> that wouldn't fit else where. In a personal discussion with [~frm], we
> figured that this class might be a good starting point refactoring
> {{segment-tar}} towards better encapsulation.
> The aim would be to return {{SegmentTracker}} to its initial purpose (i.e.
> tracking segments) and move all unrelated concerns elsewhere.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-4434) Remove segment version argument from segment writer and and related classes
[
https://issues.apache.org/jira/browse/OAK-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-4434.
Resolution: Fixed
Fixed at http://svn.apache.org/viewvc?rev=1747166&view=rev
> Remove segment version argument from segment writer and and related classes
> ---
>
> Key: OAK-4434
> URL: https://issues.apache.org/jira/browse/OAK-4434
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Michael Dürig
> Labels: refactoring
> Fix For: 1.6
>
>
> The {{SegmentWriter}} and its related classes accept a {{SegmentVersion}}
> argument. This is confusing since that version is only stored in the
> segment's segment version field. The writer cannot and does not actually
> write segments at older version than the latest (12).
> I suggest we remove the explicit segment version from all classes where it
> can be specified and hard code the segment version to 12 for now. This is the
> only segment version {{segment-tar}} currently supports anyway. Should the
> need to support other segment version arise in the future, we need to decide
> at that point how to parametrise {{segment-tar}} on the segment version.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-4434) Remove segment version argument from segment writer and and related classes
Michael Dürig created OAK-4434:
--
Summary: Remove segment version argument from segment writer and
and related classes
Key: OAK-4434
URL: https://issues.apache.org/jira/browse/OAK-4434
Project: Jackrabbit Oak
Issue Type: Improvement
Components: segment-tar
Reporter: Michael Dürig
Assignee: Michael Dürig
Fix For: 1.6
The {{SegmentWriter}} and its related classes accept a {{SegmentVersion}}
argument. This is confusing since that version is only stored in the segment's
segment version field. The writer cannot and does not actually write segments
at older version than the latest (12).
I suggest we remove the explicit segment version from all classes where it can
be specified and hard code the segment version to 12 for now. This is the only
segment version {{segment-tar}} currently supports anyway. Should the need to
support other segment version arise in the future, we need to decide at that
point how to parametrise {{segment-tar}} on the segment version.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
