[jira] [Created] (OAK-5980) Bad Join Query Plan Used
Thomas Mueller created OAK-5980:
---
Summary: Bad Join Query Plan Used
Key: OAK-5980
URL: https://issues.apache.org/jira/browse/OAK-5980
Project: Jackrabbit Oak
Issue Type: Bug
Components: query
Reporter: Thomas Mueller
Assignee: Thomas Mueller
Fix For: 1.8
For a join query, where selectors are joined over ischildnode but also can use
an index,
the selectors sometimes use the index instead of the much less
expensive parent join. Example:
{noformat}
select [a].* from [nt:unstructured] as [a]
inner join [nt:unstructured] as [b] on ischildnode([b], [a])
inner join [nt:unstructured] as [c] on ischildnode([c], [b])
inner join [nt:unstructured] as [d] on ischildnode([d], [c])
inner join [nt:unstructured] as [e] on ischildnode([e], [d])
where [a].[classname] = 'letter'
and isdescendantnode([a], '/content')
and [c].[classname] = 'chapter'
and localname([b]) = 'chapters'
and [e].[classname] = 'list'
and localname([d]) = 'lists'
and [e].[path] = cast('/content/abc' as path)
{noformat}
The order of selectors is sometimes wrong (not e, d, c, b, a), but
more importantly, selectors c and a use the index on className.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OAK-5981) Oak-Segment version check with disabled mmaping
Alex Parvulescu created OAK-5981: Summary: Oak-Segment version check with disabled mmaping Key: OAK-5981 URL: https://issues.apache.org/jira/browse/OAK-5981 Project: Jackrabbit Oak Issue Type: Technical task Reporter: Alex Parvulescu -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5981) Oak-Segment version check with disabled mmaping
[ https://issues.apache.org/jira/browse/OAK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-5981: - Fix Version/s: 1.6.2 > Oak-Segment version check with disabled mmaping > --- > > Key: OAK-5981 > URL: https://issues.apache.org/jira/browse/OAK-5981 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: run, segment-tar >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.7.0, 1.6.2 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5979) FileStore version check should disable memory mapping
[ https://issues.apache.org/jira/browse/OAK-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-5979: - Component/s: run > FileStore version check should disable memory mapping > - > > Key: OAK-5979 > URL: https://issues.apache.org/jira/browse/OAK-5979 > Project: Jackrabbit Oak > Issue Type: Bug > Components: run, segment-tar, segmentmk >Affects Versions: 1.4.15, 1.6.1 >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Labels: candidate_oak_1_4, candidate_oak_1_6 > Fix For: 1.7.0 > > > As > [discussed|https://lists.apache.org/thread.html/6a3f6cbc9169cfb70e56557fc49acf4ee62e37badc3f09a9b558fd7e@%3Coak-dev.jackrabbit.apache.org%3E] > on oak-dev list, using memory mapping for this check can cause issues with > collecting the unreferenced files, effectively doubling the size of the > virtual memory mapped for the repository. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5981) Oak-Segment version check with disabled mmaping
[ https://issues.apache.org/jira/browse/OAK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-5981: - Component/s: (was: segmentmk) run > Oak-Segment version check with disabled mmaping > --- > > Key: OAK-5981 > URL: https://issues.apache.org/jira/browse/OAK-5981 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: run, segment-tar >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.7.0, 1.6.2 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (OAK-5981) Oak-Segment version check with disabled mmaping
[ https://issues.apache.org/jira/browse/OAK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu reassigned OAK-5981: Assignee: Alex Parvulescu > Oak-Segment version check with disabled mmaping > --- > > Key: OAK-5981 > URL: https://issues.apache.org/jira/browse/OAK-5981 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: run, segment-tar >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.7.0, 1.6.2 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (OAK-5981) Oak-Segment version check with disabled mmaping
[ https://issues.apache.org/jira/browse/OAK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu resolved OAK-5981. -- Resolution: Fixed fixed on trunk with http://svn.apache.org/viewvc?rev=1788378&view=rev and on 1.6 with http://svn.apache.org/viewvc?rev=1788379&view=rev these changes are only on oak-run and affect a single command, the datastore consistency check. > Oak-Segment version check with disabled mmaping > --- > > Key: OAK-5981 > URL: https://issues.apache.org/jira/browse/OAK-5981 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: run, segment-tar >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.7.0, 1.6.2 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5981) SegmentTar version check with disabled mmaping
[ https://issues.apache.org/jira/browse/OAK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-5981: - Summary: SegmentTar version check with disabled mmaping (was: Oak-Segment version check with disabled mmaping) > SegmentTar version check with disabled mmaping > -- > > Key: OAK-5981 > URL: https://issues.apache.org/jira/browse/OAK-5981 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: run, segment-tar >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.7.0, 1.6.2 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (OAK-5982) SegmentMk version check with disabled mmaping
Alex Parvulescu created OAK-5982: Summary: SegmentMk version check with disabled mmaping Key: OAK-5982 URL: https://issues.apache.org/jira/browse/OAK-5982 Project: Jackrabbit Oak Issue Type: Technical task Components: run, segmentmk Reporter: Alex Parvulescu Assignee: Alex Parvulescu -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5982) SegmentMk version check with disabled mmaping
[ https://issues.apache.org/jira/browse/OAK-5982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-5982: - Fix Version/s: (was: 1.7.0) 1.6.2 1.4.15 > SegmentMk version check with disabled mmaping > - > > Key: OAK-5982 > URL: https://issues.apache.org/jira/browse/OAK-5982 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: run, segmentmk >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.4.15, 1.6.2 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (OAK-5982) SegmentMk version check with disabled mmaping
[ https://issues.apache.org/jira/browse/OAK-5982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu resolved OAK-5982. -- Resolution: Fixed Changes are only on oak-run. Due to the fact that SegmentMk was removed from trunk I was obligated to fix the branches directly (1.6 and 1.4, backporting was also not an option): fixed on 1.6 http://svn.apache.org/viewvc?rev=1788382&view=rev fixed on 1.4 http://svn.apache.org/viewvc?rev=1788384&view=rev > SegmentMk version check with disabled mmaping > - > > Key: OAK-5982 > URL: https://issues.apache.org/jira/browse/OAK-5982 > Project: Jackrabbit Oak > Issue Type: Technical task > Components: run, segmentmk >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.4.15, 1.6.2 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (OAK-5979) FileStore version check should disable memory mapping
[ https://issues.apache.org/jira/browse/OAK-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu resolved OAK-5979. -- Resolution: Fixed subtasks are done and backported > FileStore version check should disable memory mapping > - > > Key: OAK-5979 > URL: https://issues.apache.org/jira/browse/OAK-5979 > Project: Jackrabbit Oak > Issue Type: Bug > Components: run, segment-tar, segmentmk >Affects Versions: 1.4.15, 1.6.1 >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.7.0 > > > As > [discussed|https://lists.apache.org/thread.html/6a3f6cbc9169cfb70e56557fc49acf4ee62e37badc3f09a9b558fd7e@%3Coak-dev.jackrabbit.apache.org%3E] > on oak-dev list, using memory mapping for this check can cause issues with > collecting the unreferenced files, effectively doubling the size of the > virtual memory mapped for the repository. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5979) FileStore version check should disable memory mapping
[ https://issues.apache.org/jira/browse/OAK-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Parvulescu updated OAK-5979: - Labels: (was: candidate_oak_1_4 candidate_oak_1_6) > FileStore version check should disable memory mapping > - > > Key: OAK-5979 > URL: https://issues.apache.org/jira/browse/OAK-5979 > Project: Jackrabbit Oak > Issue Type: Bug > Components: run, segment-tar, segmentmk >Affects Versions: 1.4.15, 1.6.1 >Reporter: Alex Parvulescu >Assignee: Alex Parvulescu > Fix For: 1.7.0 > > > As > [discussed|https://lists.apache.org/thread.html/6a3f6cbc9169cfb70e56557fc49acf4ee62e37badc3f09a9b558fd7e@%3Coak-dev.jackrabbit.apache.org%3E] > on oak-dev list, using memory mapping for this check can cause issues with > collecting the unreferenced files, effectively doubling the size of the > virtual memory mapped for the repository. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OAK-4933) Create a data store implementation that integrates with Microsoft Azure Blob Storage
[ https://issues.apache.org/jira/browse/OAK-4933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940057#comment-15940057 ] Amit Jain commented on OAK-4933: [~rhudea] Thanks a lot for your contribution! Thanks [~mattvryan] for inital work and review. Committed the changes with some minor modifications (those are required for s3 as well). * oak-blob-cloud-azure - http://svn.apache.org/viewvc?rev=1788387&view=rev * oak-it changes - http://svn.apache.org/viewvc?rev=1788388&view=rev * oak-run changes - http://svn.apache.org/viewvc?rev=1788389&view=rev > Create a data store implementation that integrates with Microsoft Azure Blob > Storage > > > Key: OAK-4933 > URL: https://issues.apache.org/jira/browse/OAK-4933 > Project: Jackrabbit Oak > Issue Type: Epic > Components: blob, core >Reporter: Matt Ryan >Assignee: Amit Jain > Attachments: OAK-4933-v5.patch > > > This epic proposes the creation of a new type of Oak data store, > AzureDataStore, that offers an integration between Oak and Microsoft Azure > Blob Storage. AzureDataStore would be very similar in purpose and > functionality to S3DataStore, with a different backend that uses Azure Blob > Storage instead of S3. > Some initial exploration into this concept can be seen in my github here: > https://github.com/mattvryan/jackrabbit-oak/tree/azure-blob-store > More info about Azure Blob Storage: > * [Java > SDK|https://azure.microsoft.com/en-us/documentation/articles/storage-java-how-to-use-blob-storage/] > * [Javadoc|http://azure.github.io/azure-storage-java/] > * [Source|https://github.com/azure/azure-storage-java] (GitHub) - Microsoft's > Azure Storage Java SDK is open source and Apache licensed > * [Package > info|https://mvnrepository.com/artifact/com.microsoft.azure/azure-storage] on > mvnrepository.com > As I see it, the following work would be required: > * Create an AzureDataStore class that extends CachingDataStore > * Create a new backend for Azure Blob Storage > * Comprehensive unit testing of the new data store and backend classes > * Create test "mocks" for the necessary Azure Storage classes to facilitate > unit testing (they are all final classes and cannot be mocked directly) > * Create SharedAzureDataStore class > * Create AzureDataStoreService class > * Implement similar JMX metrics as exist for S3DataStore > * Combine and refactor existing Oak code with newly added code to make best > reuse of existing code, etc. > * Integration testing with system configured with AzureDataStore, comparison > w/ S3DataStore in terms of performance and correctness > * Modify Azure Storage SDK code to make it into a valid OSGi bundle, and > submit these changes back to that project (currently it is not an OSGi bundle > and therefore currently has to be embedded) > List isn't purported to be comprehensive of course. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (OAK-4933) Create a data store implementation that integrates with Microsoft Azure Blob Storage
[ https://issues.apache.org/jira/browse/OAK-4933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Jain resolved OAK-4933. Resolution: Fixed Fix Version/s: 1.8 1.7.0 > Create a data store implementation that integrates with Microsoft Azure Blob > Storage > > > Key: OAK-4933 > URL: https://issues.apache.org/jira/browse/OAK-4933 > Project: Jackrabbit Oak > Issue Type: Epic > Components: blob, core >Reporter: Matt Ryan >Assignee: Amit Jain > Fix For: 1.7.0, 1.8 > > Attachments: OAK-4933-v5.patch > > > This epic proposes the creation of a new type of Oak data store, > AzureDataStore, that offers an integration between Oak and Microsoft Azure > Blob Storage. AzureDataStore would be very similar in purpose and > functionality to S3DataStore, with a different backend that uses Azure Blob > Storage instead of S3. > Some initial exploration into this concept can be seen in my github here: > https://github.com/mattvryan/jackrabbit-oak/tree/azure-blob-store > More info about Azure Blob Storage: > * [Java > SDK|https://azure.microsoft.com/en-us/documentation/articles/storage-java-how-to-use-blob-storage/] > * [Javadoc|http://azure.github.io/azure-storage-java/] > * [Source|https://github.com/azure/azure-storage-java] (GitHub) - Microsoft's > Azure Storage Java SDK is open source and Apache licensed > * [Package > info|https://mvnrepository.com/artifact/com.microsoft.azure/azure-storage] on > mvnrepository.com > As I see it, the following work would be required: > * Create an AzureDataStore class that extends CachingDataStore > * Create a new backend for Azure Blob Storage > * Comprehensive unit testing of the new data store and backend classes > * Create test "mocks" for the necessary Azure Storage classes to facilitate > unit testing (they are all final classes and cannot be mocked directly) > * Create SharedAzureDataStore class > * Create AzureDataStoreService class > * Implement similar JMX metrics as exist for S3DataStore > * Combine and refactor existing Oak code with newly added code to make best > reuse of existing code, etc. > * Integration testing with system configured with AzureDataStore, comparison > w/ S3DataStore in terms of performance and correctness > * Modify Azure Storage SDK code to make it into a valid OSGi bundle, and > submit these changes back to that project (currently it is not an OSGi bundle > and therefore currently has to be embedded) > List isn't purported to be comprehensive of course. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5971) Offline compaction corrupts the journal
[
https://issues.apache.org/jira/browse/OAK-5971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrei Dulceanu updated OAK-5971:
-
Attachment: OAK-5971-02.patch
I came up with a simple test case in which a blob is added, then removed and
offline compaction is called. The test asserts there's only one valid entry in
the journal and that the size after compaction is less than the size before
compaction.
[~alexparvulescu], could you take a look at it, please?
> Offline compaction corrupts the journal
> ---
>
> Key: OAK-5971
> URL: https://issues.apache.org/jira/browse/OAK-5971
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Alex Parvulescu
>Assignee: Andrei Dulceanu
> Fix For: 1.7.0, 1.8
>
> Attachments: OAK-5971-02.patch, OAK-5971.patch
>
>
> Seems offline compaction corrupts the journal by persisting an illegal value
> as the head state.
> Pre Compaction:
> {noformat}
> ~ head segmentstore/journal.log
> 244b31cd-031d-48f2-ac64-d196ebccb43a:625 root 1490200945396
> accca99a-8829-4e16-a158-c75c08d92252:1580 root 1490200951316
> b188d865-9e26-4263-a9f1-5db3de9e8bfa:1724 root 1490200955354
> {noformat}
> Post Compaction:
> {noformat}
> ~ head segmentstore/journal.log
> org.apache.jackrabbit.oak.segment.file.JournalEntry@f2680686 root
> 1490201167938
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OAK-5983) BlobGC should log the amount of space reclaimed after GC run is done
Chetan Mehrotra created OAK-5983:
Summary: BlobGC should log the amount of space reclaimed after GC
run is done
Key: OAK-5983
URL: https://issues.apache.org/jira/browse/OAK-5983
Project: Jackrabbit Oak
Issue Type: Improvement
Components: blob
Reporter: Chetan Mehrotra
Fix For: 1.8
Currently BlobGC logic reports the number of blobs removed
{noformat}
24.03.2017 12:33:33.120 *INFO* [sling-oak-observation-205]
org.apache.jackrabbit.oak.plugins.blob.MarkSweepGarbageCollector Blob garbage
collection completed in 16.76 s. Number of blobs deleted [1776] with max
modification time of [2017-03-23 12:33:16.373]
{noformat}
It would helpful it can also report the size reclaimed by deleted those blobs
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OAK-5983) BlobGC should log the amount of space reclaimed after GC run is done
[
https://issues.apache.org/jira/browse/OAK-5983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940189#comment-15940189
]
Chetan Mehrotra commented on OAK-5983:
--
Currently in all cases blobId also encode the blob length. So it should be
possible to get this information without any extra cost. We can use the
{{BlobStore#getBlobLength}} for that or introduce a new method which gurantees
that length is calculated without doing any remote call.
[~amitjain] Thoughts?
> BlobGC should log the amount of space reclaimed after GC run is done
>
>
> Key: OAK-5983
> URL: https://issues.apache.org/jira/browse/OAK-5983
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: blob
>Reporter: Chetan Mehrotra
> Fix For: 1.8
>
>
> Currently BlobGC logic reports the number of blobs removed
> {noformat}
> 24.03.2017 12:33:33.120 *INFO* [sling-oak-observation-205]
> org.apache.jackrabbit.oak.plugins.blob.MarkSweepGarbageCollector Blob garbage
> collection completed in 16.76 s. Number of blobs deleted [1776] with max
> modification time of [2017-03-23 12:33:16.373]
> {noformat}
> It would helpful it can also report the size reclaimed by deleted those blobs
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OAK-5520) Improve index and query documentation
[ https://issues.apache.org/jira/browse/OAK-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940192#comment-15940192 ] Thomas Mueller commented on OAK-5520: - http://svn.apache.org/r1788415: Troubleshooting > Improve index and query documentation > - > > Key: OAK-5520 > URL: https://issues.apache.org/jira/browse/OAK-5520 > Project: Jackrabbit Oak > Issue Type: Bug > Components: lucene, query >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Fix For: 1.7.0 > > > The Oak index and query documentation needs to be improved: > * step-by-step descriptions of how to fix problems (for example from slow > queries to fast queries) > * checklists > * index corruption vs merely _perceive_ corruption > * checkpoints > * text extraction > * link to tools, such as the Oak Lucene Index Definition Generator at > http://oakutils.appspot.com/generate/index > * indexing and reindex: when it is needed and how to do it, how long does it > take, how to stop > * document currently undocumented features (for example Lucene index, > notNullCheckEnabled) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OAK-5528) leaseUpdateThread might be blocked by leaseUpdateCheck
[
https://issues.apache.org/jira/browse/OAK-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940216#comment-15940216
]
Julian Reschke commented on OAK-5528:
-
Good question, will check.
> leaseUpdateThread might be blocked by leaseUpdateCheck
> --
>
> Key: OAK-5528
> URL: https://issues.apache.org/jira/browse/OAK-5528
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Affects Versions: 1.4, 1.5.14
>Reporter: Stefan Eissing
>Assignee: Julian Reschke
> Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4
> Fix For: 1.6.0
>
> Attachments: OAK-5446.diff, OAK-5446-jr.diff, [#OAK-5446]
> leaseUpdateThread might be blocked by leaseUpdateCheck.html,
> OAK-5446.testcase, OAK-5446.testcase.v3, OAK-5446.xml
>
>
> {color:red}
> cloned from OAK-5446 due to internal JIRA issues
> {color}
> Fighting with cluster nodes losing their lease and shutting down oak-core in
> a cloud environment. For reasons unknown at this point in time, the whole
> process seems to skip about two minutes of real time.
> This is a situation from which oak currently does not recover. Code analysis
> shows that {{ClusterNodeInfo}} is handed the
> {{LeaseCheckDocumentStoreWrapper}} instance to use as store. This is fatal
> since any action the {{renewLease()}} tries to do will first invoke the
> {{performLeaseCheck()}}. The lease check will, when the {{FailureMargin}} is
> reached, _stall the renewLease() thread_ for 5 retry attempts and then
> declare the lease to be lost.
> The {{ClusterNodeInfo}} should instead be using the "real" {{DocumentStore}},
> not the wrapped one, IMO.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-4859) Warn if lease update is invoked with large delay
[ https://issues.apache.org/jira/browse/OAK-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4859: Labels: candidate_oak_1_4 (was: ) > Warn if lease update is invoked with large delay > > > Key: OAK-4859 > URL: https://issues.apache.org/jira/browse/OAK-4859 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Affects Versions: 1.5.10 >Reporter: Stefan Egli >Assignee: Stefan Egli > Labels: candidate_oak_1_4 > Fix For: 1.5.13, 1.6.0 > > > DocumentMk's lease mechanism is built on the assumption that the lease is > periodically updated by each instance. If the update doesn't happen within a > certain time - and the instance hasn't crashed - there's the risk of the own > lease to fail. It is therefore important that the lease update happens > without (large) delay according to the configured period. > One pattern where this doesn't happen is when the VM is under heavy load due > to JVM-Full-GC cycles. It seems likely that a memory problem doesn't normally > happen instantly, but slowly builds up. Based on this assumption we could > introduce a check that compares the actual time since last lease update with > the desired period. If these two diverge _a lot_ then we can at least issue a > log.warn. This might help to easier identify this type of lease failures and > perhaps find root causes earlier/easier. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (OAK-5528) leaseUpdateThread might be blocked by leaseUpdateCheck
[
https://issues.apache.org/jira/browse/OAK-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940216#comment-15940216
]
Julian Reschke edited comment on OAK-5528 at 3/24/17 12:29 PM:
---
bq. Are they any code dependencies that requires us to backport more changes?
Good point. Turns out we should backport OAK-4859 first; I'll assume I can go
ahead with that as it only affects logging.
was (Author: reschke):
Good question, will check.
> leaseUpdateThread might be blocked by leaseUpdateCheck
> --
>
> Key: OAK-5528
> URL: https://issues.apache.org/jira/browse/OAK-5528
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Affects Versions: 1.4, 1.5.14
>Reporter: Stefan Eissing
>Assignee: Julian Reschke
> Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4
> Fix For: 1.6.0
>
> Attachments: OAK-5446.diff, OAK-5446-jr.diff, [#OAK-5446]
> leaseUpdateThread might be blocked by leaseUpdateCheck.html,
> OAK-5446.testcase, OAK-5446.testcase.v3, OAK-5446.xml
>
>
> {color:red}
> cloned from OAK-5446 due to internal JIRA issues
> {color}
> Fighting with cluster nodes losing their lease and shutting down oak-core in
> a cloud environment. For reasons unknown at this point in time, the whole
> process seems to skip about two minutes of real time.
> This is a situation from which oak currently does not recover. Code analysis
> shows that {{ClusterNodeInfo}} is handed the
> {{LeaseCheckDocumentStoreWrapper}} instance to use as store. This is fatal
> since any action the {{renewLease()}} tries to do will first invoke the
> {{performLeaseCheck()}}. The lease check will, when the {{FailureMargin}} is
> reached, _stall the renewLease() thread_ for 5 retry attempts and then
> declare the lease to be lost.
> The {{ClusterNodeInfo}} should instead be using the "real" {{DocumentStore}},
> not the wrapped one, IMO.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5222) Optimize the multiplexing node store
[
https://issues.apache.org/jira/browse/OAK-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek updated OAK-5222:
---
Fix Version/s: 1.7.0
> Optimize the multiplexing node store
>
>
> Key: OAK-5222
> URL: https://issues.apache.org/jira/browse/OAK-5222
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.7.0, 1.8
>
> Attachments: OAK-5222-1.patch, results1.png
>
>
> I added support for the multiplexing node store for the oak-run benchmark
> fixtures. It seems that the performance in ReadDeepTreeTest is linearly
> dependent on the mount counts:
> {noformat}
> MountsN
> 0 729
> 1 402
> 2 287
> 3 209
> 4 188
> 5 154
> 6 133
> 7 126
> 8 104
> 9 100
> 1085
> 2046
> {noformat}
> {{N}} - throughput in 60 seconds. Following command was used:
> {noformat}
> $ java -jar target/oak-run-1.6-SNAPSHOT.jar benchmark ReadDeepTreeTest
> Oak-Multiplexing --mounts=10
> {noformat}
> It should be possible to improve this, so we won't have a noticeable
> performance decrease for 1000 mounts.
> //cc: [~rombert]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (OAK-5222) Optimize the multiplexing node store
[
https://issues.apache.org/jira/browse/OAK-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomek Rękawek resolved OAK-5222.
Resolution: Fixed
> Optimize the multiplexing node store
>
>
> Key: OAK-5222
> URL: https://issues.apache.org/jira/browse/OAK-5222
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.8
>
> Attachments: OAK-5222-1.patch, results1.png
>
>
> I added support for the multiplexing node store for the oak-run benchmark
> fixtures. It seems that the performance in ReadDeepTreeTest is linearly
> dependent on the mount counts:
> {noformat}
> MountsN
> 0 729
> 1 402
> 2 287
> 3 209
> 4 188
> 5 154
> 6 133
> 7 126
> 8 104
> 9 100
> 1085
> 2046
> {noformat}
> {{N}} - throughput in 60 seconds. Following command was used:
> {noformat}
> $ java -jar target/oak-run-1.6-SNAPSHOT.jar benchmark ReadDeepTreeTest
> Oak-Multiplexing --mounts=10
> {noformat}
> It should be possible to improve this, so we won't have a noticeable
> performance decrease for 1000 mounts.
> //cc: [~rombert]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-4859) Warn if lease update is invoked with large delay
[ https://issues.apache.org/jira/browse/OAK-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4859: Labels: candidate_oak_1_0 candidate_oak_1_2 candidate_oak_1_4 (was: candidate_oak_1_4) > Warn if lease update is invoked with large delay > > > Key: OAK-4859 > URL: https://issues.apache.org/jira/browse/OAK-4859 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Affects Versions: 1.5.10 >Reporter: Stefan Egli >Assignee: Stefan Egli > Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4 > Fix For: 1.5.13, 1.6.0 > > > DocumentMk's lease mechanism is built on the assumption that the lease is > periodically updated by each instance. If the update doesn't happen within a > certain time - and the instance hasn't crashed - there's the risk of the own > lease to fail. It is therefore important that the lease update happens > without (large) delay according to the configured period. > One pattern where this doesn't happen is when the VM is under heavy load due > to JVM-Full-GC cycles. It seems likely that a memory problem doesn't normally > happen instantly, but slowly builds up. Based on this assumption we could > introduce a check that compares the actual time since last lease update with > the desired period. If these two diverge _a lot_ then we can at least issue a > log.warn. This might help to easier identify this type of lease failures and > perhaps find root causes earlier/easier. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-4770) Missing exception handling in ClusterNodeInfo.renewLease()
[
https://issues.apache.org/jira/browse/OAK-4770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-4770:
Labels: candidate_oak_1_0 candidate_oak_1_2 resilience (was: resilience)
> Missing exception handling in ClusterNodeInfo.renewLease()
> --
>
> Key: OAK-4770
> URL: https://issues.apache.org/jira/browse/OAK-4770
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core, documentmk
>Reporter: Marcel Reutegger
>Assignee: Marcel Reutegger
> Labels: candidate_oak_1_0, candidate_oak_1_2, resilience
> Fix For: 1.4.10, 1.5.10, 1.6.0
>
>
> ClusterNodeInfo.renewLease() does not handle a potential
> DocumentStoreException on {{findAndModify()}}. This may leave
> {{previousLeaseEndTime}} in an inconsistent state and a subsequent
> {{renewLease()}} call then considers the lease timed out.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OAK-5971) Offline compaction corrupts the journal
[
https://issues.apache.org/jira/browse/OAK-5971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940267#comment-15940267
]
Alex Parvulescu commented on OAK-5971:
--
bq .I came up with a simple test case in which a blob is added, then removed
and offline compaction is called. The test asserts there's only one valid entry
in the journal and that the size after compaction is less than the size before
compaction.
Oh, that's too much overhead for this specific issue (I can't see how the size
of the repo is relevant here). You can augment an existing offline compaction
test and simply focus on checking the journal is not getting corrupted (ie.
start with CompactionAndCleanupIT#offlineCompaction).
> Offline compaction corrupts the journal
> ---
>
> Key: OAK-5971
> URL: https://issues.apache.org/jira/browse/OAK-5971
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Alex Parvulescu
>Assignee: Andrei Dulceanu
> Fix For: 1.7.0, 1.8
>
> Attachments: OAK-5971-02.patch, OAK-5971.patch
>
>
> Seems offline compaction corrupts the journal by persisting an illegal value
> as the head state.
> Pre Compaction:
> {noformat}
> ~ head segmentstore/journal.log
> 244b31cd-031d-48f2-ac64-d196ebccb43a:625 root 1490200945396
> accca99a-8829-4e16-a158-c75c08d92252:1580 root 1490200951316
> b188d865-9e26-4263-a9f1-5db3de9e8bfa:1724 root 1490200955354
> {noformat}
> Post Compaction:
> {noformat}
> ~ head segmentstore/journal.log
> org.apache.jackrabbit.oak.segment.file.JournalEntry@f2680686 root
> 1490201167938
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-4859) Warn if lease update is invoked with large delay
[ https://issues.apache.org/jira/browse/OAK-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4859: Fix Version/s: 1.4.15 > Warn if lease update is invoked with large delay > > > Key: OAK-4859 > URL: https://issues.apache.org/jira/browse/OAK-4859 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Affects Versions: 1.5.10 >Reporter: Stefan Egli >Assignee: Stefan Egli > Labels: candidate_oak_1_0, candidate_oak_1_2 > Fix For: 1.4.15, 1.5.13, 1.6.0 > > > DocumentMk's lease mechanism is built on the assumption that the lease is > periodically updated by each instance. If the update doesn't happen within a > certain time - and the instance hasn't crashed - there's the risk of the own > lease to fail. It is therefore important that the lease update happens > without (large) delay according to the configured period. > One pattern where this doesn't happen is when the VM is under heavy load due > to JVM-Full-GC cycles. It seems likely that a memory problem doesn't normally > happen instantly, but slowly builds up. Based on this assumption we could > introduce a check that compares the actual time since last lease update with > the desired period. If these two diverge _a lot_ then we can at least issue a > log.warn. This might help to easier identify this type of lease failures and > perhaps find root causes earlier/easier. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-4859) Warn if lease update is invoked with large delay
[ https://issues.apache.org/jira/browse/OAK-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4859: Labels: candidate_oak_1_0 candidate_oak_1_2 (was: candidate_oak_1_0 candidate_oak_1_2 candidate_oak_1_4) > Warn if lease update is invoked with large delay > > > Key: OAK-4859 > URL: https://issues.apache.org/jira/browse/OAK-4859 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Affects Versions: 1.5.10 >Reporter: Stefan Egli >Assignee: Stefan Egli > Labels: candidate_oak_1_0, candidate_oak_1_2 > Fix For: 1.4.15, 1.5.13, 1.6.0 > > > DocumentMk's lease mechanism is built on the assumption that the lease is > periodically updated by each instance. If the update doesn't happen within a > certain time - and the instance hasn't crashed - there's the risk of the own > lease to fail. It is therefore important that the lease update happens > without (large) delay according to the configured period. > One pattern where this doesn't happen is when the VM is under heavy load due > to JVM-Full-GC cycles. It seems likely that a memory problem doesn't normally > happen instantly, but slowly builds up. Based on this assumption we could > introduce a check that compares the actual time since last lease update with > the desired period. If these two diverge _a lot_ then we can at least issue a > log.warn. This might help to easier identify this type of lease failures and > perhaps find root causes earlier/easier. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OAK-4859) Warn if lease update is invoked with large delay
[ https://issues.apache.org/jira/browse/OAK-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940280#comment-15940280 ] Julian Reschke commented on OAK-4859: - trunk: [r1767502|http://svn.apache.org/r1767502] 1.4: [r1788434|http://svn.apache.org/r1788434] > Warn if lease update is invoked with large delay > > > Key: OAK-4859 > URL: https://issues.apache.org/jira/browse/OAK-4859 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: documentmk >Affects Versions: 1.5.10 >Reporter: Stefan Egli >Assignee: Stefan Egli > Labels: candidate_oak_1_0, candidate_oak_1_2 > Fix For: 1.4.15, 1.5.13, 1.6.0 > > > DocumentMk's lease mechanism is built on the assumption that the lease is > periodically updated by each instance. If the update doesn't happen within a > certain time - and the instance hasn't crashed - there's the risk of the own > lease to fail. It is therefore important that the lease update happens > without (large) delay according to the configured period. > One pattern where this doesn't happen is when the VM is under heavy load due > to JVM-Full-GC cycles. It seems likely that a memory problem doesn't normally > happen instantly, but slowly builds up. Based on this assumption we could > introduce a check that compares the actual time since last lease update with > the desired period. If these two diverge _a lot_ then we can at least issue a > log.warn. This might help to easier identify this type of lease failures and > perhaps find root causes earlier/easier. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (OAK-5528) leaseUpdateThread might be blocked by leaseUpdateCheck
[
https://issues.apache.org/jira/browse/OAK-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940216#comment-15940216
]
Julian Reschke edited comment on OAK-5528 at 3/24/17 1:19 PM:
--
bq. Are they any code dependencies that requires us to backport more changes?
Good point. Turns out we should backport OAK-4859 first; I'll assume I can go
ahead with that as it only affects logging.
Another tiny conflict is due to OAK-4184, which I believe we do not want to
backport?
was (Author: reschke):
bq. Are they any code dependencies that requires us to backport more changes?
Good point. Turns out we should backport OAK-4859 first; I'll assume I can go
ahead with that as it only affects logging.
> leaseUpdateThread might be blocked by leaseUpdateCheck
> --
>
> Key: OAK-5528
> URL: https://issues.apache.org/jira/browse/OAK-5528
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Affects Versions: 1.4, 1.5.14
>Reporter: Stefan Eissing
>Assignee: Julian Reschke
> Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4
> Fix For: 1.6.0
>
> Attachments: OAK-5446.diff, OAK-5446-jr.diff, [#OAK-5446]
> leaseUpdateThread might be blocked by leaseUpdateCheck.html,
> OAK-5446.testcase, OAK-5446.testcase.v3, OAK-5446.xml
>
>
> {color:red}
> cloned from OAK-5446 due to internal JIRA issues
> {color}
> Fighting with cluster nodes losing their lease and shutting down oak-core in
> a cloud environment. For reasons unknown at this point in time, the whole
> process seems to skip about two minutes of real time.
> This is a situation from which oak currently does not recover. Code analysis
> shows that {{ClusterNodeInfo}} is handed the
> {{LeaseCheckDocumentStoreWrapper}} instance to use as store. This is fatal
> since any action the {{renewLease()}} tries to do will first invoke the
> {{performLeaseCheck()}}. The lease check will, when the {{FailureMargin}} is
> reached, _stall the renewLease() thread_ for 5 retry attempts and then
> declare the lease to be lost.
> The {{ClusterNodeInfo}} should instead be using the "real" {{DocumentStore}},
> not the wrapped one, IMO.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OAK-5971) Offline compaction corrupts the journal
[
https://issues.apache.org/jira/browse/OAK-5971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940336#comment-15940336
]
Andrei Dulceanu commented on OAK-5971:
--
bq. I can't see how the size of the repo is relevant here.
Right, I was thinking the same, but since I was checking the validity of the
journal, I said to check also the size after compaction. I can remove the size
check to simplify the test case.
bq. You can augment an existing offline compaction test and simply focus on
checking the journal is not getting corrupted.
Initially I tried the same, but then I realised that the bug happened only for
calls to {{Compact#compact}} [1] which rewrites the {{journal}} to contain only
the head revision. Augmenting an offline compaction IT won't prevent breaking
{{oak-run compact}} since it deals only with updating the {{journal}} from
{{TarRevisions#doFlush}} [2]. In turn this is successfully tested by
{{TarRevisionsTest}} [3].
[1]
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/tool/Compact.java#L123
[2]
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/TarRevisions.java#L215
[3]
https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/file/TarRevisionsTest.java
> Offline compaction corrupts the journal
> ---
>
> Key: OAK-5971
> URL: https://issues.apache.org/jira/browse/OAK-5971
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
>Reporter: Alex Parvulescu
>Assignee: Andrei Dulceanu
> Fix For: 1.7.0, 1.8
>
> Attachments: OAK-5971-02.patch, OAK-5971.patch
>
>
> Seems offline compaction corrupts the journal by persisting an illegal value
> as the head state.
> Pre Compaction:
> {noformat}
> ~ head segmentstore/journal.log
> 244b31cd-031d-48f2-ac64-d196ebccb43a:625 root 1490200945396
> accca99a-8829-4e16-a158-c75c08d92252:1580 root 1490200951316
> b188d865-9e26-4263-a9f1-5db3de9e8bfa:1724 root 1490200955354
> {noformat}
> Post Compaction:
> {noformat}
> ~ head segmentstore/journal.log
> org.apache.jackrabbit.oak.segment.file.JournalEntry@f2680686 root
> 1490201167938
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5528) leaseUpdateThread might be blocked by leaseUpdateCheck
[
https://issues.apache.org/jira/browse/OAK-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5528:
Fix Version/s: 1.4.15
> leaseUpdateThread might be blocked by leaseUpdateCheck
> --
>
> Key: OAK-5528
> URL: https://issues.apache.org/jira/browse/OAK-5528
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Affects Versions: 1.4, 1.5.14
>Reporter: Stefan Eissing
>Assignee: Julian Reschke
> Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.4.15, 1.6.0
>
> Attachments: OAK-5446.diff, OAK-5446-jr.diff, [#OAK-5446]
> leaseUpdateThread might be blocked by leaseUpdateCheck.html,
> OAK-5446.testcase, OAK-5446.testcase.v3, OAK-5446.xml
>
>
> {color:red}
> cloned from OAK-5446 due to internal JIRA issues
> {color}
> Fighting with cluster nodes losing their lease and shutting down oak-core in
> a cloud environment. For reasons unknown at this point in time, the whole
> process seems to skip about two minutes of real time.
> This is a situation from which oak currently does not recover. Code analysis
> shows that {{ClusterNodeInfo}} is handed the
> {{LeaseCheckDocumentStoreWrapper}} instance to use as store. This is fatal
> since any action the {{renewLease()}} tries to do will first invoke the
> {{performLeaseCheck()}}. The lease check will, when the {{FailureMargin}} is
> reached, _stall the renewLease() thread_ for 5 retry attempts and then
> declare the lease to be lost.
> The {{ClusterNodeInfo}} should instead be using the "real" {{DocumentStore}},
> not the wrapped one, IMO.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5528) leaseUpdateThread might be blocked by leaseUpdateCheck
[
https://issues.apache.org/jira/browse/OAK-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5528:
Labels: candidate_oak_1_0 candidate_oak_1_2 (was: candidate_oak_1_0
candidate_oak_1_2 candidate_oak_1_4)
> leaseUpdateThread might be blocked by leaseUpdateCheck
> --
>
> Key: OAK-5528
> URL: https://issues.apache.org/jira/browse/OAK-5528
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Affects Versions: 1.4, 1.5.14
>Reporter: Stefan Eissing
>Assignee: Julian Reschke
> Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.4.15, 1.6.0
>
> Attachments: OAK-5446.diff, OAK-5446-jr.diff, [#OAK-5446]
> leaseUpdateThread might be blocked by leaseUpdateCheck.html,
> OAK-5446.testcase, OAK-5446.testcase.v3, OAK-5446.xml
>
>
> {color:red}
> cloned from OAK-5446 due to internal JIRA issues
> {color}
> Fighting with cluster nodes losing their lease and shutting down oak-core in
> a cloud environment. For reasons unknown at this point in time, the whole
> process seems to skip about two minutes of real time.
> This is a situation from which oak currently does not recover. Code analysis
> shows that {{ClusterNodeInfo}} is handed the
> {{LeaseCheckDocumentStoreWrapper}} instance to use as store. This is fatal
> since any action the {{renewLease()}} tries to do will first invoke the
> {{performLeaseCheck()}}. The lease check will, when the {{FailureMargin}} is
> reached, _stall the renewLease() thread_ for 5 retry attempts and then
> declare the lease to be lost.
> The {{ClusterNodeInfo}} should instead be using the "real" {{DocumentStore}},
> not the wrapped one, IMO.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OAK-5528) leaseUpdateThread might be blocked by leaseUpdateCheck
[
https://issues.apache.org/jira/browse/OAK-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940338#comment-15940338
]
Julian Reschke commented on OAK-5528:
-
trunk: [r1780424|http://svn.apache.org/r1780424]
1.4: [r1788441|http://svn.apache.org/r1788441]
> leaseUpdateThread might be blocked by leaseUpdateCheck
> --
>
> Key: OAK-5528
> URL: https://issues.apache.org/jira/browse/OAK-5528
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Affects Versions: 1.4, 1.5.14
>Reporter: Stefan Eissing
>Assignee: Julian Reschke
> Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.4.15, 1.6.0
>
> Attachments: OAK-5446.diff, OAK-5446-jr.diff, [#OAK-5446]
> leaseUpdateThread might be blocked by leaseUpdateCheck.html,
> OAK-5446.testcase, OAK-5446.testcase.v3, OAK-5446.xml
>
>
> {color:red}
> cloned from OAK-5446 due to internal JIRA issues
> {color}
> Fighting with cluster nodes losing their lease and shutting down oak-core in
> a cloud environment. For reasons unknown at this point in time, the whole
> process seems to skip about two minutes of real time.
> This is a situation from which oak currently does not recover. Code analysis
> shows that {{ClusterNodeInfo}} is handed the
> {{LeaseCheckDocumentStoreWrapper}} instance to use as store. This is fatal
> since any action the {{renewLease()}} tries to do will first invoke the
> {{performLeaseCheck()}}. The lease check will, when the {{FailureMargin}} is
> reached, _stall the renewLease() thread_ for 5 retry attempts and then
> declare the lease to be lost.
> The {{ClusterNodeInfo}} should instead be using the "real" {{DocumentStore}},
> not the wrapped one, IMO.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Issue Comment Deleted] (OAK-5528) leaseUpdateThread might be blocked by leaseUpdateCheck
[
https://issues.apache.org/jira/browse/OAK-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5528:
Comment: was deleted
(was: trunk: [r1780424|http://svn.apache.org/r1780424]
)
> leaseUpdateThread might be blocked by leaseUpdateCheck
> --
>
> Key: OAK-5528
> URL: https://issues.apache.org/jira/browse/OAK-5528
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: core
>Affects Versions: 1.4, 1.5.14
>Reporter: Stefan Eissing
>Assignee: Julian Reschke
> Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.4.15, 1.6.0
>
> Attachments: OAK-5446.diff, OAK-5446-jr.diff, [#OAK-5446]
> leaseUpdateThread might be blocked by leaseUpdateCheck.html,
> OAK-5446.testcase, OAK-5446.testcase.v3, OAK-5446.xml
>
>
> {color:red}
> cloned from OAK-5446 due to internal JIRA issues
> {color}
> Fighting with cluster nodes losing their lease and shutting down oak-core in
> a cloud environment. For reasons unknown at this point in time, the whole
> process seems to skip about two minutes of real time.
> This is a situation from which oak currently does not recover. Code analysis
> shows that {{ClusterNodeInfo}} is handed the
> {{LeaseCheckDocumentStoreWrapper}} instance to use as store. This is fatal
> since any action the {{renewLease()}} tries to do will first invoke the
> {{performLeaseCheck()}}. The lease check will, when the {{FailureMargin}} is
> reached, _stall the renewLease() thread_ for 5 retry attempts and then
> declare the lease to be lost.
> The {{ClusterNodeInfo}} should instead be using the "real" {{DocumentStore}},
> not the wrapped one, IMO.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OAK-5978) Allow deployers to configure a query time limit
[ https://issues.apache.org/jira/browse/OAK-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940361#comment-15940361 ] Thomas Mueller commented on OAK-5978: - How is your request different from what we have already? As Davide wrote on the Oak list, we cancel a query that iterated over a certain amount of nodes or as of 1.6 may hit the traversal index (doesn't have an index). As for resource exhaustion: we have a limit on the number of nodes read in memory. The number of traversed nodes is very similar to a time limit: the more nodes are traversed, the slower a query is (it's basically a 1:1 relationship). Measuring the time taken for a query is tricky, we would have to measure each individual "nextNode()" call, and sum that. This would include time to load binaries from Lucene indexes (possibly reads from the S3 datastore). This is highly dependant on caching, much more than a typical query in a relational database with a local file system. > Allow deployers to configure a query time limit > --- > > Key: OAK-5978 > URL: https://issues.apache.org/jira/browse/OAK-5978 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: query >Affects Versions: 1.6.1 >Reporter: Ian Boston > > Currently when a query takes a long time to complete, Oak allows it to > continue to completion, which can under certain circumstances result in > resource exhaustion and slow performance for all operations depending on Oak. > This feature request is to apply a deployer configurable time limit to all > queries. If the query exceeds that time limit, it is canceled with a suitable > exception, so that it consumes no more resources and the code or user that > submitted the query is notified. > Ideally this needs to work while the query is executing inside Oak rather > than waiting to return via the Oak API before being canceled. Cancelation > should result in minimal resource usage. > At this stage, this is an idea. It may not be possible to implement. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OAK-5978) Allow deployers to configure a query time limit
[ https://issues.apache.org/jira/browse/OAK-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940365#comment-15940365 ] Thomas Mueller commented on OAK-5978: - See also http://jackrabbit.apache.org/oak/docs/query/query-engine.html#Slow_Queries_and_Read_Limits (for LimitReads and LimitInMemory) and http://jackrabbit.apache.org/oak/docs/query/query-engine.html#Query_Options (for QueryEngineSettings.failTraversal) > Allow deployers to configure a query time limit > --- > > Key: OAK-5978 > URL: https://issues.apache.org/jira/browse/OAK-5978 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: query >Affects Versions: 1.6.1 >Reporter: Ian Boston > > Currently when a query takes a long time to complete, Oak allows it to > continue to completion, which can under certain circumstances result in > resource exhaustion and slow performance for all operations depending on Oak. > This feature request is to apply a deployer configurable time limit to all > queries. If the query exceeds that time limit, it is canceled with a suitable > exception, so that it consumes no more resources and the code or user that > submitted the query is notified. > Ideally this needs to work while the query is executing inside Oak rather > than waiting to return via the Oak API before being canceled. Cancelation > should result in minimal resource usage. > At this stage, this is an idea. It may not be possible to implement. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5972) Provide a way to abort indexing / reindexing of synchronous indexes
[ https://issues.apache.org/jira/browse/OAK-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Mueller updated OAK-5972: Summary: Provide a way to abort indexing / reindexing of synchronous indexes (was: Provide a way to abort indexing / reindexing) > Provide a way to abort indexing / reindexing of synchronous indexes > --- > > Key: OAK-5972 > URL: https://issues.apache.org/jira/browse/OAK-5972 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: query >Reporter: Thomas Mueller > Fix For: 1.8 > > > Indexing and reindexing is slow. There should be a way to abort this (for > example if it was started by accident). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OAK-5972) Provide a way to abort indexing / reindexing of synchronous indexes
[ https://issues.apache.org/jira/browse/OAK-5972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940412#comment-15940412 ] Thomas Mueller commented on OAK-5972: - [~chetanm] this is for synchronous indexes. Right now it's part of the commit, but once we split up reindexing it will be possible to stop that as well. (Or maybe we just use the same / a similar mechanism as for asynchronous reindexing, in which case we have solved that problem as well). > Provide a way to abort indexing / reindexing of synchronous indexes > --- > > Key: OAK-5972 > URL: https://issues.apache.org/jira/browse/OAK-5972 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: query >Reporter: Thomas Mueller > Fix For: 1.8 > > > Indexing and reindexing is slow. There should be a way to abort this (for > example if it was started by accident). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OAK-5978) Allow deployers to configure a query time limit
[ https://issues.apache.org/jira/browse/OAK-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940424#comment-15940424 ] Ian Boston commented on OAK-5978: - It's different for the reasons you stated. When a query causes the working set of an active repository to increase beyond the resources that are available, everything slows down. That shows up as page faulting in TarMK, page faults with MongoMK on MMAPv1 and saturated IO in MongoMK WT. Counting the number of results or looking for traversal is no longer a good way of detecting a query that should be canceled. Starting a timer at the start of the query and checking the time elapsed at convenient points throughout the query execution would be a better way of protecting normal operation from roque queries. That suggestion is based on my very limited knowledge of how Oak works internally. There are probably much better ways of doing it based on elapsed time. Perhaps it can't be implemented in Oak. The current limits don't appear to protect against outage. > Allow deployers to configure a query time limit > --- > > Key: OAK-5978 > URL: https://issues.apache.org/jira/browse/OAK-5978 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: query >Affects Versions: 1.6.1 >Reporter: Ian Boston > > Currently when a query takes a long time to complete, Oak allows it to > continue to completion, which can under certain circumstances result in > resource exhaustion and slow performance for all operations depending on Oak. > This feature request is to apply a deployer configurable time limit to all > queries. If the query exceeds that time limit, it is canceled with a suitable > exception, so that it consumes no more resources and the code or user that > submitted the query is notified. > Ideally this needs to work while the query is executing inside Oak rather > than waiting to return via the Oak API before being canceled. Cancelation > should result in minimal resource usage. > At this stage, this is an idea. It may not be possible to implement. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OAK-5978) Allow deployers to configure a query time limit
[ https://issues.apache.org/jira/browse/OAK-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940437#comment-15940437 ] Thomas Mueller commented on OAK-5978: - > The current limits don't appear to protect against outage. Could you provide more details about that please. > Starting a timer at the start of the query and checking the time elapsed at > convenient points Well, there are different times limits, which one are you interested in: * The time since the query was started (for example, between the Query.execute() and the very latest call for this query, which might be NodeIterator.nextNode() or for example getSize()). This is relatively easy to measure. * The accumulated time of each call for the given query. That is, the time needed for Query.execute(), plus the time needed for each individual NodeIterator.nextNode(), and all other calls individually). > Allow deployers to configure a query time limit > --- > > Key: OAK-5978 > URL: https://issues.apache.org/jira/browse/OAK-5978 > Project: Jackrabbit Oak > Issue Type: New Feature > Components: query >Affects Versions: 1.6.1 >Reporter: Ian Boston > > Currently when a query takes a long time to complete, Oak allows it to > continue to completion, which can under certain circumstances result in > resource exhaustion and slow performance for all operations depending on Oak. > This feature request is to apply a deployer configurable time limit to all > queries. If the query exceeds that time limit, it is canceled with a suitable > exception, so that it consumes no more resources and the code or user that > submitted the query is notified. > Ideally this needs to work while the query is executing inside Oak rather > than waiting to return via the Oak API before being canceled. Cancelation > should result in minimal resource usage. > At this stage, this is an idea. It may not be possible to implement. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5855:
Attachment: (was: OAK-5855-3.diff)
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff,
> OAK-5855-9.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5855:
Attachment: (was: OAK-5855-8.diff)
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff,
> OAK-5855-9.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5855:
Attachment: OAK-5855-16.diff
(wip) logging
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff,
> OAK-5855-9.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5855:
Attachment: (was: OAK-5855-6.diff)
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff,
> OAK-5855-9.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5855:
Attachment: (was: OAK-5855.diff)
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff,
> OAK-5855-9.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5855:
Attachment: (was: OAK-5855-2.diff)
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff,
> OAK-5855-9.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5855:
Attachment: (was: OAK-5855-7.diff)
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff,
> OAK-5855-9.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5855:
Attachment: (was: OAK-5855-4.diff)
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff,
> OAK-5855-9.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5855:
Attachment: (was: OAK-5855-5.diff)
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff,
> OAK-5855-9.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-5855:
Attachment: (was: OAK-5855-9.diff)
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OAK-5984) Property indexes can get ouf of sync
Thomas Mueller created OAK-5984: --- Summary: Property indexes can get ouf of sync Key: OAK-5984 URL: https://issues.apache.org/jira/browse/OAK-5984 Project: Jackrabbit Oak Issue Type: Improvement Components: query Reporter: Thomas Mueller Assignee: Thomas Mueller Fix For: 1.8 Property indexes can get out of sync for the following reasons: * the index was disabled for some time * the property index component was not started / configured * the index definition was changed without reindexing -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (OAK-5985) add CloseableIterator similar to CloseableIterable
Julian Reschke created OAK-5985: --- Summary: add CloseableIterator similar to CloseableIterable Key: OAK-5985 URL: https://issues.apache.org/jira/browse/OAK-5985 Project: Jackrabbit Oak Issue Type: Improvement Components: commons Reporter: Julian Reschke -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5985) add CloseableIterator similar to CloseableIterable
[ https://issues.apache.org/jira/browse/OAK-5985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-5985: Fix Version/s: 1.8 > add CloseableIterator similar to CloseableIterable > -- > > Key: OAK-5985 > URL: https://issues.apache.org/jira/browse/OAK-5985 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: commons >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.8 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (OAK-5985) add CloseableIterator similar to CloseableIterable
[ https://issues.apache.org/jira/browse/OAK-5985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke reassigned OAK-5985: --- Assignee: Julian Reschke > add CloseableIterator similar to CloseableIterable > -- > > Key: OAK-5985 > URL: https://issues.apache.org/jira/browse/OAK-5985 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: commons >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.8 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5985) add CloseableIterator similar to CloseableIterable
[ https://issues.apache.org/jira/browse/OAK-5985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-5985: Description: Needed by OAK-5855 but tracked separately so it can be independently back-ported. > add CloseableIterator similar to CloseableIterable > -- > > Key: OAK-5985 > URL: https://issues.apache.org/jira/browse/OAK-5985 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: commons >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.8 > > > Needed by OAK-5855 but tracked separately so it can be independently > back-ported. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5985) add CloseableIterator similar to CloseableIterable
[ https://issues.apache.org/jira/browse/OAK-5985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-5985: Fix Version/s: 1.7.0 > add CloseableIterator similar to CloseableIterable > -- > > Key: OAK-5985 > URL: https://issues.apache.org/jira/browse/OAK-5985 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: commons >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.7.0, 1.8 > > > Needed by OAK-5855 but tracked separately so it can be independently > back-ported. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (OAK-5985) add CloseableIterator similar to CloseableIterable
[ https://issues.apache.org/jira/browse/OAK-5985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-5985. - Resolution: Fixed > add CloseableIterator similar to CloseableIterable > -- > > Key: OAK-5985 > URL: https://issues.apache.org/jira/browse/OAK-5985 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: commons >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.7.0, 1.8 > > > Needed by OAK-5855 but tracked separately so it can be independently > back-ported. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OAK-5985) add CloseableIterator similar to CloseableIterable
[ https://issues.apache.org/jira/browse/OAK-5985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940504#comment-15940504 ] Julian Reschke commented on OAK-5985: - trunk: [r1788463|http://svn.apache.org/r1788463] > add CloseableIterator similar to CloseableIterable > -- > > Key: OAK-5985 > URL: https://issues.apache.org/jira/browse/OAK-5985 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: commons >Reporter: Julian Reschke >Assignee: Julian Reschke > Fix For: 1.7.0, 1.8 > > > Needed by OAK-5855 but tracked separately so it can be independently > back-ported. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke resolved OAK-5855.
-
Resolution: Fixed
Fix Version/s: 1.8
1.7.0
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Fix For: 1.7.0, 1.8
>
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OAK-5855) RDBDocumentStore: improve query support for VersionGC
[
https://issues.apache.org/jira/browse/OAK-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940585#comment-15940585
]
Julian Reschke commented on OAK-5855:
-
trunk: [r1788476|http://svn.apache.org/r1788476]
> RDBDocumentStore: improve query support for VersionGC
> -
>
> Key: OAK-5855
> URL: https://issues.apache.org/jira/browse/OAK-5855
> Project: Jackrabbit Oak
> Issue Type: Technical task
> Components: rdbmk
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Minor
> Fix For: 1.7.0, 1.8
>
> Attachments: OAK-5855-10.diff, OAK-5855-11.diff, OAK-5855-12.diff,
> OAK-5855-13.diff, OAK-5855-14.diff, OAK-5855-15.diff, OAK-5855-16.diff
>
>
> ...matching {{MongoVersionSupport}}, such as: no batched query needed; just
> return an iterable over delete candidates.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (OAK-5986) Clarify default nodestore/datastore in http://jackrabbit.apache.org/oak/docs/osgi_config.html
Konrad Windszus created OAK-5986: Summary: Clarify default nodestore/datastore in http://jackrabbit.apache.org/oak/docs/osgi_config.html Key: OAK-5986 URL: https://issues.apache.org/jira/browse/OAK-5986 Project: Jackrabbit Oak Issue Type: Improvement Components: doc Reporter: Konrad Windszus Although with OAK-3159 the documentation around segment stores have been greatly improved it is still not clear which segment store is configured if no explicit configuration is found (in OSGi). Please add that information for both the data store as well as the node store. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-4780) VersionGarbageCollector should be able to run incrementally
[ https://issues.apache.org/jira/browse/OAK-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-4780: Attachment: OAJK-4780-rdb.diff RDB-specific extensions, not completely in sync with Stefan's generic/mogo patches though. > VersionGarbageCollector should be able to run incrementally > --- > > Key: OAK-4780 > URL: https://issues.apache.org/jira/browse/OAK-4780 > Project: Jackrabbit Oak > Issue Type: Task > Components: core, documentmk >Reporter: Julian Reschke > Attachments: leafnodes.diff, leafnodes-v2.diff, leafnodes-v3.diff, > OAJK-4780-rdb.diff > > > Right now, the documentmk's version garbage collection runs in several phases. > It first collects the paths of candidate nodes, and only once this has been > successfully finished, starts actually deleting nodes. > This can be a problem when the regularly scheduled garbage collection is > interrupted during the path collection phase, maybe due to other maintenance > tasks. On the next run, the number of paths to be collected will be even > bigger, thus making it even more likely to fail. > We should think about a change in the logic that would allow the GC to run in > chunks; maybe by partitioning the path space by top level directory. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-3342) move benchmarks in oak-benchmark module
[ https://issues.apache.org/jira/browse/OAK-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella updated OAK-3342: -- Attachment: OAK-3342-1.diff [patch|^OAK-3342-1.diff] for review. The build compiles fine and oak-run went from 54MB to 34MB and we added cloud-azure module since last build. Still the pending questions about backports remain. Don't really know what to do. If prefer to review individual commits, see the github link in a previous comment. > move benchmarks in oak-benchmark module > --- > > Key: OAK-3342 > URL: https://issues.apache.org/jira/browse/OAK-3342 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: run >Reporter: Davide Giannella >Assignee: Davide Giannella >Priority: Minor > Fix For: 1.8 > > Attachments: OAK-3342-1.diff > > > Take all the benchmarks provided by oak-run and move them into the > oak-development module. Micro-benchmarking and Scalability -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (OAK-5762) [Oak Lucene] Several full-text operators do not work (NOT, !, AND)
[
https://issues.apache.org/jira/browse/OAK-5762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Gonzalez updated OAK-5762:
Summary: [Oak Lucene] Several full-text operators do not work (NOT, !, AND)
(was: [Oak Lucene] NOT and ! full-text operators do not work.)
> [Oak Lucene] Several full-text operators do not work (NOT, !, AND)
> --
>
> Key: OAK-5762
> URL: https://issues.apache.org/jira/browse/OAK-5762
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.0
>Reporter: David Gonzalez
>
> The following 3 full-text queries return the same results. I expected the 1st
> to be different from the 2nd and 3rd, but the 2nd and 3rd should be identical
> to one another. All 3 yield the same results.
> {noformat}
> dog AND cat
> {noformat}
> {noformat}
> dog AND NOT cat
> {noformat}
> {noformat}
> dog AND !cat
> {noformat}
> The following (which, IIUC is the equivalent to NOT and !) does yield
> expected results.
> {noformat}
> dog AND -cat
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (OAK-5762) [Oak Lucene] Several full-text operators do not work (NOT, !, AND)
[
https://issues.apache.org/jira/browse/OAK-5762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Gonzalez updated OAK-5762:
Description:
The following fulltext operators do not appear to be evaluated correctly.
AND operator
{noformat}
QUERY: /jcr:root/content/docs/en/aem/_x0036_-3//element(*,
cq:Page)[(jcr:contains(., 'response AND layout'))]/rep:excerpt(.)
PLAN: [cq:Page] as [a] /* lucene:cqPageLucene(/oak:index/cqPageLucene)
+(+:fulltext:respons +:fulltext:and +:fulltext:layout)
+:ancestors:/content/docs/en/aem/6-3 ft:("response" "AND" "layout") where
(contains([a].[*], 'response AND layout')) and (isdescendantnode([a],
[/content/docs/en/aem/6-3])) */
{noformat}
NOT operator
{noformat}
QUERY: /jcr:root/content/docs/en/aem/_x0036_-3//element(*,
cq:Page)[(jcr:contains(., 'response NOT layout'))]/rep:excerpt(.)
PLAN: [cq:Page] as [a] /* lucene:cqPageLucene(/oak:index/cqPageLucene)
+(+:fulltext:respons +:fulltext:not +:fulltext:layout)
+:ancestors:/content/docs/en/aem/6-3 ft:("response" "NOT" "layout") where
(contains([a].[*], 'response NOT layout')) and (isdescendantnode([a],
[/content/docs/en/aem/6-3])) */
{noformat}
! operator
{noformat}
QUERY: /jcr:root/content/docs/en/aem/_x0036_-3//element(*,
cq:Page)[(jcr:contains(., 'response !layout'))]/rep:excerpt(.)
PLAN: [cq:Page] as [a] /* lucene:cqPageLucene(/oak:index/cqPageLucene)
+(+:fulltext:respons +:fulltext:layout) +:ancestors:/content/docs/en/aem/6-3
ft:("response" "!layout") where (contains([a].[*], 'response !layout')) and
(isdescendantnode([a], [/content/docs/en/aem/6-3])) */
{noformat}
Note the `-` operator works.
was:
The following 3 full-text queries return the same results. I expected the 1st
to be different from the 2nd and 3rd, but the 2nd and 3rd should be identical
to one another. All 3 yield the same results.
{noformat}
dog AND cat
{noformat}
{noformat}
dog AND NOT cat
{noformat}
{noformat}
dog AND !cat
{noformat}
The following (which, IIUC is the equivalent to NOT and !) does yield expected
results.
{noformat}
dog AND -cat
{noformat}
> [Oak Lucene] Several full-text operators do not work (NOT, !, AND)
> --
>
> Key: OAK-5762
> URL: https://issues.apache.org/jira/browse/OAK-5762
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: lucene
>Affects Versions: 1.6.0
>Reporter: David Gonzalez
>
> The following fulltext operators do not appear to be evaluated correctly.
> AND operator
> {noformat}
> QUERY: /jcr:root/content/docs/en/aem/_x0036_-3//element(*,
> cq:Page)[(jcr:contains(., 'response AND layout'))]/rep:excerpt(.)
> PLAN: [cq:Page] as [a] /* lucene:cqPageLucene(/oak:index/cqPageLucene)
> +(+:fulltext:respons +:fulltext:and +:fulltext:layout)
> +:ancestors:/content/docs/en/aem/6-3 ft:("response" "AND" "layout") where
> (contains([a].[*], 'response AND layout')) and (isdescendantnode([a],
> [/content/docs/en/aem/6-3])) */
> {noformat}
> NOT operator
> {noformat}
> QUERY: /jcr:root/content/docs/en/aem/_x0036_-3//element(*,
> cq:Page)[(jcr:contains(., 'response NOT layout'))]/rep:excerpt(.)
> PLAN: [cq:Page] as [a] /* lucene:cqPageLucene(/oak:index/cqPageLucene)
> +(+:fulltext:respons +:fulltext:not +:fulltext:layout)
> +:ancestors:/content/docs/en/aem/6-3 ft:("response" "NOT" "layout") where
> (contains([a].[*], 'response NOT layout')) and (isdescendantnode([a],
> [/content/docs/en/aem/6-3])) */
> {noformat}
> ! operator
> {noformat}
> QUERY: /jcr:root/content/docs/en/aem/_x0036_-3//element(*,
> cq:Page)[(jcr:contains(., 'response !layout'))]/rep:excerpt(.)
> PLAN: [cq:Page] as [a] /* lucene:cqPageLucene(/oak:index/cqPageLucene)
> +(+:fulltext:respons +:fulltext:layout) +:ancestors:/content/docs/en/aem/6-3
> ft:("response" "!layout") where (contains([a].[*], 'response !layout')) and
> (isdescendantnode([a], [/content/docs/en/aem/6-3])) */
> {noformat}
> Note the `-` operator works.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
