[jira] [Updated] (OAK-3110) AsyncIndexer fails due to FileNotFoundException thrown by CopyOnWrite logic
[
https://issues.apache.org/jira/browse/OAK-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-3110:
-
Attachment: copier.log
Attaching [logs|^copier.log] from IndexCopier and AsyncIndexUpdate for a single
run where the run failed due to exception in CopyOnWrite. The issue here has
its root in OAK-2855
h3. Interaction between CopyOnWrite (COW) and CopyOnRead (COR)
COW logic is implemented with the requirement that it has sure the same local
directory as used in COR. This ensures that it can make use of alread copied
index files while indexing. Any indexing sequence would work in following
sequence
# COW - Index done - Index moved to R1
# COR - Open at R1
# COW - Index moves to R2
# COR - Picks up index at R2
# COW - Starts indexing for R3
Further COR takes a snapshot of current local directory content and when it
closes it deletes those files which are present in local directory but not
present in remote directory. This ensures that issues seen in OAK-2855 do not
happen. This approach would have worked fine if above sequence is strictly
sequential. However in real world thats not the case as the 2 task are
performed via different threads.
For e.g. say at #3 when COW is done with indexing the Lucene index update
observor would detect that index has updated. This observation processing would
happen in a different thread. Now when at #4 COR is opening the index it can
happen that COW has also started with its index at #5 and hence started
creating new files. These new files would be picked by COR at #4 during its
opening phase and it would consider them part of old index. Now when COR would
close it would delete these files and that would then result in a
FileNotFoundException for COW in #5
h3. Mapping to Logs
In the attached logs following timeline is seen
{noformat}
14:20:01.700 *DEBUG* [pool-58-thread-1] AsyncIndexUpdate Running background
index task async
14:20:08.601 *DEBUG* [pool-58-thread-1] IndexCopier [COW] Creating output
_2s7.fdt
14:20:18.146 *DEBUG* [oak-lucene-0] IndexCopier [/oak:index/foxDamIndex]
Following files have been removed from Lucene index directory [_2s7.fdt,
_2s7.fdx]
14:20:35.752 *DEBUG* [oak-lucene-0] IndexCopier Error occurred while copying
files. Further processing would be skipped
java.io.FileNotFoundException: _2s7.fdt
at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:261)
14:20:35.930 *WARN* [pool-58-thread-1] AsyncIndexUpdate The async index update
failed
{noformat}
So as can be seen that COW starts at 20:01 and creates _2s7.fdt at 20:08.
However that file got deleted by oak-lucene-0 (thread which opens index with
COR) at 20:18 and then eventually FileNotFoundException is thrown when copy
logic in COW tries to read the file
h3. Fix
As a fix COW and COR would have to share the current working set and COR should
not touch files in that working set
AsyncIndexer fails due to FileNotFoundException thrown by CopyOnWrite logic
---
Key: OAK-3110
URL: https://issues.apache.org/jira/browse/OAK-3110
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Fix For: 1.2.3, 1.3.3, 1.0.18
Attachments: copier.log
At times the CopyOnWrite reports following exception
{noformat}
15.07.2015 14:20:35.930 *WARN* [pool-58-thread-1]
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate The async index
update failed
org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0004: Failed to
close the Lucene index
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:204)
at
org.apache.jackrabbit.oak.plugins.index.IndexUpdate.leave(IndexUpdate.java:219)
at
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:56)
at
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate.updateIndex(AsyncIndexUpdate.java:366)
at
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate.run(AsyncIndexUpdate.java:311)
at
org.apache.sling.commons.scheduler.impl.QuartzJobExecutor.execute(QuartzJobExecutor.java:105)
at org.quartz.core.JobRunShell.run(JobRunShell.java:207)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: _2s7.fdt
at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:261)
at
[jira] [Assigned] (OAK-3099) Revision GC fails when split documents with very long paths are present
[
https://issues.apache.org/jira/browse/OAK-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain reassigned OAK-3099:
--
Assignee: Amit Jain
Revision GC fails when split documents with very long paths are present
---
Key: OAK-3099
URL: https://issues.apache.org/jira/browse/OAK-3099
Project: Jackrabbit Oak
Issue Type: Bug
Components: mongomk
Affects Versions: 1.0.13
Reporter: Csaba Varga
Assignee: Amit Jain
Priority: Minor
Fix For: 1.2.3, 1.3.3, 1.0.18
Attachments: OAK-3099.patch, SplitDocumentGenerator.java
My company is using the MongoDB microkernel with Oak, and we've noticed that
the daily revision GC is failing with errors like this:
{code}
13.07.2015 13:06:16.261 *ERROR* [pool-7-thread-1-Maintenance
Queue(com/adobe/granite/maintenance/job/RevisionCleanupTask)]
org.apache.jackrabbit.oak.management.ManagementOperation Revision garbage
collection failed
java.lang.IllegalArgumentException:
13:h113f9d0fe7ac0f87fa06397c37b9ffd4b372eeb1ec93e0818bb4024a32587820
at
org.apache.jackrabbit.oak.plugins.document.Revision.fromString(Revision.java:236)
at
org.apache.jackrabbit.oak.plugins.document.SplitDocumentCleanUp.disconnect(SplitDocumentCleanUp.java:84)
at
org.apache.jackrabbit.oak.plugins.document.SplitDocumentCleanUp.disconnect(SplitDocumentCleanUp.java:56)
at
org.apache.jackrabbit.oak.plugins.document.VersionGCSupport.deleteSplitDocuments(VersionGCSupport.java:53)
at
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector.collectSplitDocuments(VersionGarbageCollector.java:117)
at
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector.gc(VersionGarbageCollector.java:105)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService$2.run(DocumentNodeStoreService.java:511)
at org.apache.jackrabbit.oak.spi.state.RevisionGC$1.call(RevisionGC.java:68)
at org.apache.jackrabbit.oak.spi.state.RevisionGC$1.call(RevisionGC.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}
I've narrowed the issue down to the disconnect(NodeDocument) method of the
[SplitDocumentCleanUp
class|https://svn.apache.org/repos/asf/jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/SplitDocumentCleanUp.java].
The method always tries to extract the path of the node from its ID, but
this won't work for documents whose path is very long because those documents
will have the hash of their path in the ID.
I believe this code should fix the issue, but I haven't had a chance to
actually try it:
{code}
private void disconnect(NodeDocument splitDoc) {
String mainId = Utils.getIdFromPath(splitDoc.getMainPath());
NodeDocument doc = store.find(NODES, mainId);
if (doc == null) {
LOG.warn(Main document {} already removed. Split document is {},
mainId, splitId);
return;
}
String path = splitDoc.getPath();
int slashIdx = path.lastIndexOf('/');
int height = Integer.parseInt(path.substring(slashIdx + 1));
Revision rev = Revision.fromString(
path.substring(path.lastIndexOf('/', slashIdx - 1) + 1,
slashIdx));
doc = doc.findPrevReferencingDoc(rev, height);
if (doc == null) {
LOG.warn(Split document {} not referenced anymore. Main document
is {},
splitId, mainId);
return;
}
// remove reference
if (doc.getSplitDocType() == INTERMEDIATE) {
disconnectFromIntermediate(doc, rev);
} else {
markStaleOnMain(doc, rev, height);
}
}
{code}
By using getPath(), the code should automatically use either the ID or the
_path property, whichever is right for the document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3099) Revision GC fails when split documents with very long paths are present
[
https://issues.apache.org/jira/browse/OAK-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain updated OAK-3099:
---
Fix Version/s: 1.0.18
1.3.3
1.2.3
Revision GC fails when split documents with very long paths are present
---
Key: OAK-3099
URL: https://issues.apache.org/jira/browse/OAK-3099
Project: Jackrabbit Oak
Issue Type: Bug
Components: mongomk
Affects Versions: 1.0.13
Reporter: Csaba Varga
Assignee: Amit Jain
Priority: Minor
Fix For: 1.2.3, 1.3.3, 1.0.18
Attachments: OAK-3099.patch, SplitDocumentGenerator.java
My company is using the MongoDB microkernel with Oak, and we've noticed that
the daily revision GC is failing with errors like this:
{code}
13.07.2015 13:06:16.261 *ERROR* [pool-7-thread-1-Maintenance
Queue(com/adobe/granite/maintenance/job/RevisionCleanupTask)]
org.apache.jackrabbit.oak.management.ManagementOperation Revision garbage
collection failed
java.lang.IllegalArgumentException:
13:h113f9d0fe7ac0f87fa06397c37b9ffd4b372eeb1ec93e0818bb4024a32587820
at
org.apache.jackrabbit.oak.plugins.document.Revision.fromString(Revision.java:236)
at
org.apache.jackrabbit.oak.plugins.document.SplitDocumentCleanUp.disconnect(SplitDocumentCleanUp.java:84)
at
org.apache.jackrabbit.oak.plugins.document.SplitDocumentCleanUp.disconnect(SplitDocumentCleanUp.java:56)
at
org.apache.jackrabbit.oak.plugins.document.VersionGCSupport.deleteSplitDocuments(VersionGCSupport.java:53)
at
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector.collectSplitDocuments(VersionGarbageCollector.java:117)
at
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector.gc(VersionGarbageCollector.java:105)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService$2.run(DocumentNodeStoreService.java:511)
at org.apache.jackrabbit.oak.spi.state.RevisionGC$1.call(RevisionGC.java:68)
at org.apache.jackrabbit.oak.spi.state.RevisionGC$1.call(RevisionGC.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}
I've narrowed the issue down to the disconnect(NodeDocument) method of the
[SplitDocumentCleanUp
class|https://svn.apache.org/repos/asf/jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/SplitDocumentCleanUp.java].
The method always tries to extract the path of the node from its ID, but
this won't work for documents whose path is very long because those documents
will have the hash of their path in the ID.
I believe this code should fix the issue, but I haven't had a chance to
actually try it:
{code}
private void disconnect(NodeDocument splitDoc) {
String mainId = Utils.getIdFromPath(splitDoc.getMainPath());
NodeDocument doc = store.find(NODES, mainId);
if (doc == null) {
LOG.warn(Main document {} already removed. Split document is {},
mainId, splitId);
return;
}
String path = splitDoc.getPath();
int slashIdx = path.lastIndexOf('/');
int height = Integer.parseInt(path.substring(slashIdx + 1));
Revision rev = Revision.fromString(
path.substring(path.lastIndexOf('/', slashIdx - 1) + 1,
slashIdx));
doc = doc.findPrevReferencingDoc(rev, height);
if (doc == null) {
LOG.warn(Split document {} not referenced anymore. Main document
is {},
splitId, mainId);
return;
}
// remove reference
if (doc.getSplitDocType() == INTERMEDIATE) {
disconnectFromIntermediate(doc, rev);
} else {
markStaleOnMain(doc, rev, height);
}
}
{code}
By using getPath(), the code should automatically use either the ID or the
_path property, whichever is right for the document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-3110) AsyncIndexer fails due to FileNotFoundException thrown by CopyOnWrite logic
[
https://issues.apache.org/jira/browse/OAK-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629197#comment-14629197
]
Chetan Mehrotra edited comment on OAK-3110 at 7/16/15 5:05 AM:
---
Attaching [logs|^copier.log] from IndexCopier and AsyncIndexUpdate for a single
run where the run failed due to exception in CopyOnWrite. The issue here has
its root in OAK-2855
h3. Interaction between CopyOnWrite (COW) and CopyOnRead (COR)
COW logic is implemented with the requirement that it has to *share* the same
local directory as used in COR. This ensures that it can make use of already
copied index files while indexing. Any indexing sequence would work in
following sequence
# COW - Index done - Index moved to R1
# COR - Open at R1
# COW - Index moves to R2
# COR - Picks up index at R2
# COW - Starts indexing for R3
Further COR takes a snapshot of current local directory content and when it
closes it deletes those files which are present in local directory but not
present in remote directory. This ensures that issues seen in OAK-2855 do not
happen. This approach would have worked fine if above sequence is strictly
sequential. However in real world thats not the case as the 2 tasks are
performed via different threads.
For e.g. say at #3 when COW is done with indexing the Lucene index update
observor would detect that index has updated. This observation processing would
happen in a different thread. Now when at #4 COR is opening the index it can
happen that COW has also started with its index at #5 and hence started
creating new files. These new files would be picked by COR at #4 during its
opening phase and it would consider them part of old index. Now when COR would
close it would delete these files and that would then result in a
FileNotFoundException for COW in #5
h3. Mapping to Logs
In the attached logs following timeline is seen
{noformat}
14:20:01.700 *DEBUG* [pool-58-thread-1] AsyncIndexUpdate Running background
index task async
14:20:08.601 *DEBUG* [pool-58-thread-1] IndexCopier [COW] Creating output
_2s7.fdt
14:20:18.146 *DEBUG* [oak-lucene-0] IndexCopier [/oak:index/foxDamIndex]
Following files have been removed from Lucene index directory [_2s7.fdt,
_2s7.fdx]
14:20:35.752 *DEBUG* [oak-lucene-0] IndexCopier Error occurred while copying
files. Further processing would be skipped
java.io.FileNotFoundException: _2s7.fdt
at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:261)
14:20:35.930 *WARN* [pool-58-thread-1] AsyncIndexUpdate The async index update
failed
{noformat}
So as can be seen that COW starts at 20:01 and creates _2s7.fdt at 20:08.
However that file got deleted by oak-lucene-0 (thread which opens index with
COR) at 20:18 and then eventually FileNotFoundException is thrown when copy
logic in COW tries to read the file
h3. Fix
As a fix COW and COR would have to share the current working set and COR should
not touch files in that working set
was (Author: chetanm):
Attaching [logs|^copier.log] from IndexCopier and AsyncIndexUpdate for a single
run where the run failed due to exception in CopyOnWrite. The issue here has
its root in OAK-2855
h3. Interaction between CopyOnWrite (COW) and CopyOnRead (COR)
COW logic is implemented with the requirement that it has sure the same local
directory as used in COR. This ensures that it can make use of alread copied
index files while indexing. Any indexing sequence would work in following
sequence
# COW - Index done - Index moved to R1
# COR - Open at R1
# COW - Index moves to R2
# COR - Picks up index at R2
# COW - Starts indexing for R3
Further COR takes a snapshot of current local directory content and when it
closes it deletes those files which are present in local directory but not
present in remote directory. This ensures that issues seen in OAK-2855 do not
happen. This approach would have worked fine if above sequence is strictly
sequential. However in real world thats not the case as the 2 task are
performed via different threads.
For e.g. say at #3 when COW is done with indexing the Lucene index update
observor would detect that index has updated. This observation processing would
happen in a different thread. Now when at #4 COR is opening the index it can
happen that COW has also started with its index at #5 and hence started
creating new files. These new files would be picked by COR at #4 during its
opening phase and it would consider them part of old index. Now when COR would
close it would delete these files and that would then result in a
FileNotFoundException for COW in #5
h3. Mapping to Logs
In the attached logs following timeline is seen
{noformat}
14:20:01.700 *DEBUG* [pool-58-thread-1] AsyncIndexUpdate Running background
index task async
14:20:08.601 *DEBUG* [pool-58-thread-1] IndexCopier [COW] Creating output
_2s7.fdt
14:20:18.146 *DEBUG*
[jira] [Comment Edited] (OAK-1842) ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
[
https://issues.apache.org/jira/browse/OAK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627748#comment-14627748
]
Konrad Windszus edited comment on OAK-1842 at 7/15/15 9:05 AM:
---
Great, thanks for your analysis. I am just wondering why that specific error
did not occur with the file data store but only with the FileBlobStore, if this
is caused by a bug in the SegmentWriter. Don't you store the length for all
data stores with the SegmentWriter? When I conducted a test, I used exactly the
same content within the repository and with the FileBlobStore I experience the
exception, while with the File Data Store I didn't.
was (Author: kwin):
Great, thanks for your analysis. I am just wondering why that specific error
did not occur with the file data store but only with the FileBlobStore, if this
is caused by a bug in the SegmentWriter. Don't you store the length for all
data stores with the SegmentWriter?
ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
Key: OAK-1842
URL: https://issues.apache.org/jira/browse/OAK-1842
Project: Jackrabbit Oak
Issue Type: Bug
Components: blob
Affects Versions: 1.0
Reporter: Konrad Windszus
Assignee: Francesco Mari
Labels: resilience
Fix For: 1.3.3
The stacktrace of the call shows something like
{code}
20.05.2014 11:13:07.428 *ERROR* [OsgiInstallerImpl]
com.adobe.granite.installer.factory.packages.impl.PackageTransformer Error
while processing install task.
java.lang.IllegalStateException: Unexpected value record type: f2
at
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.length(SegmentBlob.java:101)
at
org.apache.jackrabbit.oak.plugins.value.BinaryImpl.getSize(BinaryImpl.java:74)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:435)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:376)
at
org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.getPackage(JcrPackageImpl.java:324)
{code}
The blob store was configured correctly and according to the log also
correctly initialized
{code}
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService
Initializing SegmentNodeStore with BlobStore
[org.apache.jackrabbit.oak.spi.blob.FileBlobStore@7e3dec43]
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService Component
still not activated. Ignoring the initialization call
20.05.2014 11:11:07.077 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK opened:
crx-quickstart/repository/segmentstore (mmap=true)
{code}
Under which circumstances can the length within the SegmentBlob be invalid?
This only happens if a File Blob Store is configured
(http://jackrabbit.apache.org/oak/docs/osgi_config.html). If a file datastore
is used, there is no such exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3106) DocumentStorePerformanceTest collects duplicate ids to remove
Marcel Reutegger created OAK-3106: - Summary: DocumentStorePerformanceTest collects duplicate ids to remove Key: OAK-3106 URL: https://issues.apache.org/jira/browse/OAK-3106 Project: Jackrabbit Oak Issue Type: Bug Components: core, mongomk Affects Versions: 1.0.17, 1.3.2, 1.2.2 Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor Fix For: 1.2.3, 1.3.3, 1.0.18 The createPerf() method runs in a loop for one second and collects ids to be removed later. The ids List is never cleared and each iteration adds duplicate ids to removeMe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3103) Stale document in MongoDocumentStore cache
[
https://issues.apache.org/jira/browse/OAK-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627707#comment-14627707
]
Marcel Reutegger commented on OAK-3103:
---
The issue is with {{MongoDocumentStore.applyToCache()}} called by the
{{update()}} method. Before the actual update call to MongoDB, {{update()}}
remembers all cached documents it wants to update. After the update it calls
{{applyToCache()}} for each document it saw in the cache while holding the lock
for the document id. The lock makes sure it is the only thread working on the
cache for the given id. In {{applyToCache()}} it checks if the document is
still in the cache. If it is not there, the cache cannot be re-populated with
the cached document before the update call and the UpdateOp applied. The
document could have been modified by another thread in the meantime. If the
document is still cached, {{applyToCache()}} will create a new document, based
on the cached document, apply the UpdateOp and then make an attempt to [add it
to the cache if it is not
present|https://github.com/apache/jackrabbit-oak/blob/jackrabbit-oak-1.0.17/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/mongo/MongoDocumentStore.java#L1078].
This is where a race condition occurs. The cached document may disappear from
the cache even though the current thread holds the lock for the document id.
The cache implementation may evict the document because other reads occur in
the meantime. As a result {{applyToCache()}} may put a potentially outdated
document into the cache.
Stale document in MongoDocumentStore cache
--
Key: OAK-3103
URL: https://issues.apache.org/jira/browse/OAK-3103
Project: Jackrabbit Oak
Issue Type: Bug
Components: core, mongomk
Affects Versions: 1.2.2, 1.3.2, 1.0.17
Reporter: Marcel Reutegger
Assignee: Marcel Reutegger
In some rare cases it may happen that a stale document is put into the
MongoDocumentStore cache.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-1842) ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
[
https://issues.apache.org/jira/browse/OAK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627769#comment-14627769
]
Francesco Mari commented on OAK-1842:
-
The {{SegmentWriter}} groups blobs in three groups: small, medium and large.
Small and medium blobs are always inlined in the segment store, and the
{{BlobStore}}, if there is one, is never used. For larghe blobs, the behaviour
of the {{SegmentWriter}} differs, depending if a {{BlobStore}} is used.
If there is no {{BlobStore}}, the blob has to be stored in the TAR files. In
this case, the TAR storage systems allows very large blobs (larger than you can
actually store in your hard drive).
If a {{BlobStore}} is configured, the blob is passed to it and the {{BobStore}}
just returns a reference to it. The reference is represented by a {{String}}.
The bug affected this case, because the length of the reference string was not
validated properly before writing it.
ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
Key: OAK-1842
URL: https://issues.apache.org/jira/browse/OAK-1842
Project: Jackrabbit Oak
Issue Type: Bug
Components: blob
Affects Versions: 1.0
Reporter: Konrad Windszus
Assignee: Francesco Mari
Labels: resilience
Fix For: 1.3.3
The stacktrace of the call shows something like
{code}
20.05.2014 11:13:07.428 *ERROR* [OsgiInstallerImpl]
com.adobe.granite.installer.factory.packages.impl.PackageTransformer Error
while processing install task.
java.lang.IllegalStateException: Unexpected value record type: f2
at
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.length(SegmentBlob.java:101)
at
org.apache.jackrabbit.oak.plugins.value.BinaryImpl.getSize(BinaryImpl.java:74)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:435)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:376)
at
org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.getPackage(JcrPackageImpl.java:324)
{code}
The blob store was configured correctly and according to the log also
correctly initialized
{code}
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService
Initializing SegmentNodeStore with BlobStore
[org.apache.jackrabbit.oak.spi.blob.FileBlobStore@7e3dec43]
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService Component
still not activated. Ignoring the initialization call
20.05.2014 11:11:07.077 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK opened:
crx-quickstart/repository/segmentstore (mmap=true)
{code}
Under which circumstances can the length within the SegmentBlob be invalid?
This only happens if a File Blob Store is configured
(http://jackrabbit.apache.org/oak/docs/osgi_config.html). If a file datastore
is used, there is no such exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3103) Stale document in MongoDocumentStore cache
[ https://issues.apache.org/jira/browse/OAK-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627701#comment-14627701 ] Marcel Reutegger commented on OAK-3103: --- Added a test (currently ignored). It fails most of the time, but sometimes it even passes all 10 iterations within the test loop. In trunk: http://svn.apache.org/r1691139 Stale document in MongoDocumentStore cache -- Key: OAK-3103 URL: https://issues.apache.org/jira/browse/OAK-3103 Project: Jackrabbit Oak Issue Type: Bug Components: core, mongomk Affects Versions: 1.2.2, 1.3.2, 1.0.17 Reporter: Marcel Reutegger Assignee: Marcel Reutegger In some rare cases it may happen that a stale document is put into the MongoDocumentStore cache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3105) SegmentWriter doesn't properly check the length of external blob IDs
[
https://issues.apache.org/jira/browse/OAK-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-3105:
Priority: Critical (was: Major)
SegmentWriter doesn't properly check the length of external blob IDs
Key: OAK-3105
URL: https://issues.apache.org/jira/browse/OAK-3105
Project: Jackrabbit Oak
Issue Type: Bug
Affects Versions: 1.2.2, 1.3.2, 1.0.17
Reporter: Francesco Mari
Priority: Critical
To store the length field of an external binary ID, the following encoding is
used:
{noformat}
1110 + 4bit + 8bit
{noformat}
which allows to store numbers between 0 and 2{^}12^ - 1.
The current implementation of {{SegmentWriter}} allows the length of binary
IDs to range between 0 and 2{^}13^ - 1, writing incorrect data when the
length of the binary ID ranges from 2{^}12^ to 2{^}13^ - 1.
When reading this incorrect data back, an {{IllegalStateException}} is thrown
complaining that the first byte of the length fields has an unexpected value
record type. See OAK-1842 for an example.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3107) SegmentWriter should be able to store blob IDs longer than 4096 bytes
Francesco Mari created OAK-3107:
---
Summary: SegmentWriter should be able to store blob IDs longer
than 4096 bytes
Key: OAK-3107
URL: https://issues.apache.org/jira/browse/OAK-3107
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Francesco Mari
The {{SegmentWriter}} is able to store blob IDs that are no longer than 4096
bytes, but some implementation of {{BlobStore}} may return blob IDs 4096 bytes
long (or more).
It should be possible to use a different encoding for long blob IDs. The blob
IDs should be written as a string (using {{SegmentWriter#writeString}}), and
its reference ID embedded into a value record.
The encoding in this case should be something like the following:
{noformat}
0 + 3bit + 3byte
{noformat}
where the three least significant bits of the first bytes are actually unused,
and the three bytes are used to store the record ID of the string representing
the blob ID.
This new encoding is necessary to maintain backwards compatibility with the
current way of storing blob IDs and to give a way to {{SegmentBlob}} to
recognise this new encoding.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-3106) DocumentStorePerformanceTest collects duplicate ids to remove
[ https://issues.apache.org/jira/browse/OAK-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-3106. --- Resolution: Fixed Fixed in trunk: http://svn.apache.org/r1691151, 1.2 branch: http://svn.apache.org/r1691153 and 1.0 branch: http://svn.apache.org/r1691155 DocumentStorePerformanceTest collects duplicate ids to remove - Key: OAK-3106 URL: https://issues.apache.org/jira/browse/OAK-3106 Project: Jackrabbit Oak Issue Type: Bug Components: core, mongomk Affects Versions: 1.2.2, 1.3.2, 1.0.17 Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor Fix For: 1.2.3, 1.3.3, 1.0.18 The createPerf() method runs in a loop for one second and collects ids to be removed later. The ids List is never cleared and each iteration adds duplicate ids to removeMe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3103) Stale document in MongoDocumentStore cache
[ https://issues.apache.org/jira/browse/OAK-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627835#comment-14627835 ] Marcel Reutegger commented on OAK-3103: --- Changed MongoDocumentStore to perform an update instead of findAndModify for a conditional update and unconditionally put the document into the cache if the before document is known. This makes the test fail more likely. Done in trunk: http://svn.apache.org/r1691159 Stale document in MongoDocumentStore cache -- Key: OAK-3103 URL: https://issues.apache.org/jira/browse/OAK-3103 Project: Jackrabbit Oak Issue Type: Bug Components: core, mongomk Affects Versions: 1.2.2, 1.3.2, 1.0.17 Reporter: Marcel Reutegger Assignee: Marcel Reutegger In some rare cases it may happen that a stale document is put into the MongoDocumentStore cache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-1842) ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
[
https://issues.apache.org/jira/browse/OAK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627740#comment-14627740
]
Francesco Mari commented on OAK-1842:
-
A first step in fixing this issue is described in OAK-3105. That issue also
describes the cause of the behaviour reported here.
ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
Key: OAK-1842
URL: https://issues.apache.org/jira/browse/OAK-1842
Project: Jackrabbit Oak
Issue Type: Bug
Components: blob
Affects Versions: 1.0
Reporter: Konrad Windszus
Assignee: Francesco Mari
Labels: resilience
Fix For: 1.3.3
The stacktrace of the call shows something like
{code}
20.05.2014 11:13:07.428 *ERROR* [OsgiInstallerImpl]
com.adobe.granite.installer.factory.packages.impl.PackageTransformer Error
while processing install task.
java.lang.IllegalStateException: Unexpected value record type: f2
at
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.length(SegmentBlob.java:101)
at
org.apache.jackrabbit.oak.plugins.value.BinaryImpl.getSize(BinaryImpl.java:74)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:435)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:376)
at
org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.getPackage(JcrPackageImpl.java:324)
{code}
The blob store was configured correctly and according to the log also
correctly initialized
{code}
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService
Initializing SegmentNodeStore with BlobStore
[org.apache.jackrabbit.oak.spi.blob.FileBlobStore@7e3dec43]
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService Component
still not activated. Ignoring the initialization call
20.05.2014 11:11:07.077 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK opened:
crx-quickstart/repository/segmentstore (mmap=true)
{code}
Under which circumstances can the length within the SegmentBlob be invalid?
This only happens if a File Blob Store is configured
(http://jackrabbit.apache.org/oak/docs/osgi_config.html). If a file datastore
is used, there is no such exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-1842) ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
[
https://issues.apache.org/jira/browse/OAK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627748#comment-14627748
]
Konrad Windszus commented on OAK-1842:
--
Great, thanks for your analysis. I am just wondering why that specific error
did not occur with the file data store but only with the FileBlobStore, if this
is caused by a bug in the SegmentWriter. Don't you store the length for all
data stores with the SegmentWriter?
ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
Key: OAK-1842
URL: https://issues.apache.org/jira/browse/OAK-1842
Project: Jackrabbit Oak
Issue Type: Bug
Components: blob
Affects Versions: 1.0
Reporter: Konrad Windszus
Assignee: Francesco Mari
Labels: resilience
Fix For: 1.3.3
The stacktrace of the call shows something like
{code}
20.05.2014 11:13:07.428 *ERROR* [OsgiInstallerImpl]
com.adobe.granite.installer.factory.packages.impl.PackageTransformer Error
while processing install task.
java.lang.IllegalStateException: Unexpected value record type: f2
at
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.length(SegmentBlob.java:101)
at
org.apache.jackrabbit.oak.plugins.value.BinaryImpl.getSize(BinaryImpl.java:74)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:435)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:376)
at
org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.getPackage(JcrPackageImpl.java:324)
{code}
The blob store was configured correctly and according to the log also
correctly initialized
{code}
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService
Initializing SegmentNodeStore with BlobStore
[org.apache.jackrabbit.oak.spi.blob.FileBlobStore@7e3dec43]
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService Component
still not activated. Ignoring the initialization call
20.05.2014 11:11:07.077 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK opened:
crx-quickstart/repository/segmentstore (mmap=true)
{code}
Under which circumstances can the length within the SegmentBlob be invalid?
This only happens if a File Blob Store is configured
(http://jackrabbit.apache.org/oak/docs/osgi_config.html). If a file datastore
is used, there is no such exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-1842) ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
[
https://issues.apache.org/jira/browse/OAK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627803#comment-14627803
]
Chetan Mehrotra commented on OAK-1842:
--
I think thats because the blobId for a FileDataStore is always 40 chars long
while for FileBlobStore it can be variable depending on number of chunks
ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
Key: OAK-1842
URL: https://issues.apache.org/jira/browse/OAK-1842
Project: Jackrabbit Oak
Issue Type: Bug
Components: blob
Affects Versions: 1.0
Reporter: Konrad Windszus
Assignee: Francesco Mari
Labels: resilience
Fix For: 1.3.3
The stacktrace of the call shows something like
{code}
20.05.2014 11:13:07.428 *ERROR* [OsgiInstallerImpl]
com.adobe.granite.installer.factory.packages.impl.PackageTransformer Error
while processing install task.
java.lang.IllegalStateException: Unexpected value record type: f2
at
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.length(SegmentBlob.java:101)
at
org.apache.jackrabbit.oak.plugins.value.BinaryImpl.getSize(BinaryImpl.java:74)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:435)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:376)
at
org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.getPackage(JcrPackageImpl.java:324)
{code}
The blob store was configured correctly and according to the log also
correctly initialized
{code}
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService
Initializing SegmentNodeStore with BlobStore
[org.apache.jackrabbit.oak.spi.blob.FileBlobStore@7e3dec43]
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService Component
still not activated. Ignoring the initialization call
20.05.2014 11:11:07.077 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK opened:
crx-quickstart/repository/segmentstore (mmap=true)
{code}
Under which circumstances can the length within the SegmentBlob be invalid?
This only happens if a File Blob Store is configured
(http://jackrabbit.apache.org/oak/docs/osgi_config.html). If a file datastore
is used, there is no such exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3105) SegmentWriter doesn't properly check the length of external blob IDs
Francesco Mari created OAK-3105:
---
Summary: SegmentWriter doesn't properly check the length of
external blob IDs
Key: OAK-3105
URL: https://issues.apache.org/jira/browse/OAK-3105
Project: Jackrabbit Oak
Issue Type: Bug
Affects Versions: 1.0.17, 1.3.2, 1.2.2
Reporter: Francesco Mari
To store the length field of an external binary ID, the following encoding is
used:
{noformat}
1110 + 4bit + 8bit
{noformat}
which allows to store numbers between 0 and 2{^}12^ - 1.
The current implementation of {{SegmentWriter}} allows the length of binary IDs
to range between 0 and 2{^}13^ - 1, writing incorrect data when the length of
the binary ID ranges from 2{^}12^ to 2{^}13^ - 1.
When reading this incorrect data back, an {{IllegalStateException}} is thrown
complaining that the first byte of the length fields has an unexpected value
record type. See OAK-1842 for an example.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3105) SegmentWriter doesn't properly check the length of external blob IDs
[
https://issues.apache.org/jira/browse/OAK-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-3105:
Attachment: OAK-3105-01.patch
The attached patch fixes the validation of the length of blob IDs. How the blob
IDs are written in the segment is not changed, to maintain backwards
compatibility.
SegmentWriter doesn't properly check the length of external blob IDs
Key: OAK-3105
URL: https://issues.apache.org/jira/browse/OAK-3105
Project: Jackrabbit Oak
Issue Type: Bug
Affects Versions: 1.2.2, 1.3.2, 1.0.17
Reporter: Francesco Mari
Priority: Critical
Attachments: OAK-3105-01.patch
To store the length field of an external binary ID, the following encoding is
used:
{noformat}
1110 + 4bit + 8bit
{noformat}
which allows to store numbers between 0 and 2{^}12^ - 1.
The current implementation of {{SegmentWriter}} allows the length of binary
IDs to range between 0 and 2{^}13^ - 1, writing incorrect data when the
length of the binary ID ranges from 2{^}12^ to 2{^}13^ - 1.
When reading this incorrect data back, an {{IllegalStateException}} is thrown
complaining that the first byte of the length fields has an unexpected value
record type. See OAK-1842 for an example.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-1842) ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
[
https://issues.apache.org/jira/browse/OAK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627748#comment-14627748
]
Konrad Windszus edited comment on OAK-1842 at 7/15/15 9:24 AM:
---
Great, thanks for your analysis. I am just wondering why that specific error
did not occur with the file data store but only with the FileBlobStore, if this
is caused by a bug in the SegmentWriter. Don't you store the length for all
blobIds of all datastores with the SegmentWriter? When I conducted a test, I
used exactly the same content within the repository and with the FileBlobStore
I experience the exception, while with the File Data Store I didn't.
How are the BlobIDs different between those two?
was (Author: kwin):
Great, thanks for your analysis. I am just wondering why that specific error
did not occur with the file data store but only with the FileBlobStore, if this
is caused by a bug in the SegmentWriter. Don't you store the length for all
data stores with the SegmentWriter? When I conducted a test, I used exactly the
same content within the repository and with the FileBlobStore I experience the
exception, while with the File Data Store I didn't.
ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
Key: OAK-1842
URL: https://issues.apache.org/jira/browse/OAK-1842
Project: Jackrabbit Oak
Issue Type: Bug
Components: blob
Affects Versions: 1.0
Reporter: Konrad Windszus
Assignee: Francesco Mari
Labels: resilience
Fix For: 1.3.3
The stacktrace of the call shows something like
{code}
20.05.2014 11:13:07.428 *ERROR* [OsgiInstallerImpl]
com.adobe.granite.installer.factory.packages.impl.PackageTransformer Error
while processing install task.
java.lang.IllegalStateException: Unexpected value record type: f2
at
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.length(SegmentBlob.java:101)
at
org.apache.jackrabbit.oak.plugins.value.BinaryImpl.getSize(BinaryImpl.java:74)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:435)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:376)
at
org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.getPackage(JcrPackageImpl.java:324)
{code}
The blob store was configured correctly and according to the log also
correctly initialized
{code}
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService
Initializing SegmentNodeStore with BlobStore
[org.apache.jackrabbit.oak.spi.blob.FileBlobStore@7e3dec43]
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService Component
still not activated. Ignoring the initialization call
20.05.2014 11:11:07.077 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK opened:
crx-quickstart/repository/segmentstore (mmap=true)
{code}
Under which circumstances can the length within the SegmentBlob be invalid?
This only happens if a File Blob Store is configured
(http://jackrabbit.apache.org/oak/docs/osgi_config.html). If a file datastore
is used, there is no such exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3102) LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
[
https://issues.apache.org/jira/browse/OAK-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627620#comment-14627620
]
Amit Jain commented on OAK-3102:
An issue already existed for the same OAK-3072. Please mark either one as
duplicate.
LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
Key: OAK-3102
URL: https://issues.apache.org/jira/browse/OAK-3102
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Priority: Minor
Fix For: 1.3.3
LuceneIndexEditorTest#copyOnWriteAndLocks is failing on Windows
{noformat}
copyOnWriteAndLocks(org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest)
Time elapsed: 0.11 sec ERROR!
org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0003: Failed to
index the node /test
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.addOrUpdate(LuceneIndexEditor.java:306)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:198)
at
org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
at
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:130)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
at
org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest.copyOnWriteAndLocks(LuceneIndexEditorTest.java:376)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.IOException: Cannot overwrite:
C:\tmp\junit308522119715585104\2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae\1_1.fdt
at org.apache.lucene.store.FSDirectory.ensureCanWrite(FSDirectory.java:293)
at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:282)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnWriteDirectory$COWLocalFileReference.createOutput(IndexCopier.java:848)
at
[jira] [Commented] (OAK-3102) LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
[
https://issues.apache.org/jira/browse/OAK-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627623#comment-14627623
]
Chetan Mehrotra commented on OAK-3102:
--
Thanks for pointing that out. Marked this one as duplicate
LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
Key: OAK-3102
URL: https://issues.apache.org/jira/browse/OAK-3102
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Priority: Minor
Fix For: 1.3.3
LuceneIndexEditorTest#copyOnWriteAndLocks is failing on Windows
{noformat}
copyOnWriteAndLocks(org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest)
Time elapsed: 0.11 sec ERROR!
org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0003: Failed to
index the node /test
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.addOrUpdate(LuceneIndexEditor.java:306)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:198)
at
org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
at
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:130)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
at
org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest.copyOnWriteAndLocks(LuceneIndexEditorTest.java:376)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.IOException: Cannot overwrite:
C:\tmp\junit308522119715585104\2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae\1_1.fdt
at org.apache.lucene.store.FSDirectory.ensureCanWrite(FSDirectory.java:293)
at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:282)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnWriteDirectory$COWLocalFileReference.createOutput(IndexCopier.java:848)
at
[jira] [Assigned] (OAK-3072) LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
[
https://issues.apache.org/jira/browse/OAK-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra reassigned OAK-3072:
Assignee: Chetan Mehrotra
LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
Key: OAK-3072
URL: https://issues.apache.org/jira/browse/OAK-3072
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Affects Versions: 1.3.1
Environment: windows
Reporter: Amit Jain
Assignee: Chetan Mehrotra
LuceneIndexEditorTest#copyOnWriteAndLocks regularly fails on windows with the
following exception
{noformat}
copyOnWriteAndLocks(org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest)
Time elapsed: 0.19 sec ERROR!
org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0003: Failed to
index the node /test
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.addOrUpdate(LuceneIndexEditor.java:297)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:191)
at
org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
at
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:130)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
at
org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest.copyOnWriteAndLocks(LuceneIndexEditorTest.java:376)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.IOException: Cannot overwrite:
C:\Users\amjain\AppData\Local\Temp\junit516545657673578972\2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae\1\_1.fdt
at
[jira] [Resolved] (OAK-3102) LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
[
https://issues.apache.org/jira/browse/OAK-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-3102.
--
Resolution: Duplicate
LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
Key: OAK-3102
URL: https://issues.apache.org/jira/browse/OAK-3102
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Priority: Minor
Fix For: 1.3.3
LuceneIndexEditorTest#copyOnWriteAndLocks is failing on Windows
{noformat}
copyOnWriteAndLocks(org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest)
Time elapsed: 0.11 sec ERROR!
org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0003: Failed to
index the node /test
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.addOrUpdate(LuceneIndexEditor.java:306)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:198)
at
org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
at
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:130)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
at
org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest.copyOnWriteAndLocks(LuceneIndexEditorTest.java:376)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.IOException: Cannot overwrite:
C:\tmp\junit308522119715585104\2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae\1_1.fdt
at org.apache.lucene.store.FSDirectory.ensureCanWrite(FSDirectory.java:293)
at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:282)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnWriteDirectory$COWLocalFileReference.createOutput(IndexCopier.java:848)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnWriteDirectory.createOutput(IndexCopier.java:618)
at
[jira] [Updated] (OAK-3072) LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
[
https://issues.apache.org/jira/browse/OAK-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-3072:
-
Priority: Minor (was: Major)
LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
Key: OAK-3072
URL: https://issues.apache.org/jira/browse/OAK-3072
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Affects Versions: 1.3.1
Environment: windows
Reporter: Amit Jain
Assignee: Chetan Mehrotra
Priority: Minor
LuceneIndexEditorTest#copyOnWriteAndLocks regularly fails on windows with the
following exception
{noformat}
copyOnWriteAndLocks(org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest)
Time elapsed: 0.19 sec ERROR!
org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0003: Failed to
index the node /test
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.addOrUpdate(LuceneIndexEditor.java:297)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:191)
at
org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
at
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:130)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
at
org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest.copyOnWriteAndLocks(LuceneIndexEditorTest.java:376)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at
org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.IOException: Cannot overwrite:
C:\Users\amjain\AppData\Local\Temp\junit516545657673578972\2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae\1\_1.fdt
[jira] [Commented] (OAK-3104) Version garbage collector doesn't collect a rolled back document if it was never deleted
[
https://issues.apache.org/jira/browse/OAK-3104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627665#comment-14627665
]
Vikas Saurabh commented on OAK-3104:
As discussed with [~mreutegg], we should probably set_deletedOnce flag while
rolling back. I'd add a test and patch later.
Version garbage collector doesn't collect a rolled back document if it was
never deleted
Key: OAK-3104
URL: https://issues.apache.org/jira/browse/OAK-3104
Project: Jackrabbit Oak
Issue Type: Bug
Components: mongomk
Reporter: Vikas Saurabh
Priority: Minor
If a commit gets rolled back it can leave (in case the document was never
deleted explicitly) a document in a state like:
{noformat}
{
_id : 7:/etc/workflow/packages/2014/10/12/rep:ours,
_deleted : {
},
_commitRoot : {
},
jcr:primaryType : {
},
_modified : NumberLong(1413126245),
_children : true,
_modCount : NumberLong(2)
}
{noformat}
If the path is fairly busy, the document can get created naturally later and
then follow the usual cycle. But, at times, such documents are ephemeral in
nature and never re-used. In those cases, such documents can remain silently
without getting collected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-2892) Speed up lucene indexing post migration by pre extracting the text content from binaries
[ https://issues.apache.org/jira/browse/OAK-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-2892. -- Resolution: Fixed Speed up lucene indexing post migration by pre extracting the text content from binaries Key: OAK-2892 URL: https://issues.apache.org/jira/browse/OAK-2892 Project: Jackrabbit Oak Issue Type: New Feature Components: lucene, run Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Labels: performance Fix For: 1.2.3, 1.3.3, 1.0.18 While migrating large repositories say having 3 M docs (250k PDF) Lucene indexing takes long time to complete (at time 4 days!). Currently the text extraction logic is coupled with Lucene indexing and hence is performed in a single threaded mode which slows down the indexing process. Further if the reindexing has to be triggered it has to be done all over again. To speed up the Lucene indexing we can decouple the text extraction from actual indexing. It is partly based on discussion on OAK-2787 # Introduce a new ExtractedTextProvider which can provide extracted text for a given Blob instance # In oak-run introduce a new indexer mode - This would take a path in repository and would then traverse the repository and look for existing binaries and extract text from that So before or after migration is done one can run this oak-run tool to create this store which has the text already extracted. Then post startup we need to wire up the ExtractedTextProvider instance (which is backed by the BlobStore populated before) and indexing logic can just get content from that. This would avoid performing expensive text extraction in the indexing thread. See discussion thread http://markmail.org/thread/ndlfpkwfgpey6o66 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3102) LuceneIndexEditorTest#copyOnWriteAndLocks failing on windows
Chetan Mehrotra created OAK-3102:
Summary: LuceneIndexEditorTest#copyOnWriteAndLocks failing on
windows
Key: OAK-3102
URL: https://issues.apache.org/jira/browse/OAK-3102
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Priority: Minor
Fix For: 1.3.3
LuceneIndexEditorTest#copyOnWriteAndLocks is failing on Windows
{noformat}
copyOnWriteAndLocks(org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest)
Time elapsed: 0.11 sec ERROR!
org.apache.jackrabbit.oak.api.CommitFailedException: OakLucene0003: Failed to
index the node /test
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.addOrUpdate(LuceneIndexEditor.java:306)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:198)
at
org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
at
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
at
org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:130)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
at org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
at
org.apache.jackrabbit.oak.spi.commit.EditorHook.processCommit(EditorHook.java:54)
at
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorTest.copyOnWriteAndLocks(LuceneIndexEditorTest.java:376)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
at org.junit.rules.RunRules.evaluate(RunRules.java:18)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.IOException: Cannot overwrite:
C:\tmp\junit308522119715585104\2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae\1_1.fdt
at org.apache.lucene.store.FSDirectory.ensureCanWrite(FSDirectory.java:293)
at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:282)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnWriteDirectory$COWLocalFileReference.createOutput(IndexCopier.java:848)
at
org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnWriteDirectory.createOutput(IndexCopier.java:618)
at
org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:44)
at
org.apache.lucene.codecs.lucene40.Lucene40StoredFieldsWriter.init(Lucene40StoredFieldsWriter.java:95)
at
[jira] [Created] (OAK-3103) Stale document in MongoDocumentStore cache
Marcel Reutegger created OAK-3103: - Summary: Stale document in MongoDocumentStore cache Key: OAK-3103 URL: https://issues.apache.org/jira/browse/OAK-3103 Project: Jackrabbit Oak Issue Type: Bug Components: core, mongomk Affects Versions: 1.0.17, 1.3.2, 1.2.2 Reporter: Marcel Reutegger Assignee: Marcel Reutegger In some rare cases it may happen that a stale document is put into the MongoDocumentStore cache. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3104) Version garbage collector doesn't collect a rolled back document if it was never deleted
Vikas Saurabh created OAK-3104:
--
Summary: Version garbage collector doesn't collect a rolled back
document if it was never deleted
Key: OAK-3104
URL: https://issues.apache.org/jira/browse/OAK-3104
Project: Jackrabbit Oak
Issue Type: Bug
Components: mongomk
Reporter: Vikas Saurabh
Priority: Minor
If a commit gets rolled back it can leave (in case the document was never
deleted explicitly) a document in a state like:
{noformat}
{
_id : 7:/etc/workflow/packages/2014/10/12/rep:ours,
_deleted : {
},
_commitRoot : {
},
jcr:primaryType : {
},
_modified : NumberLong(1413126245),
_children : true,
_modCount : NumberLong(2)
}
{noformat}
If the path is fairly busy, the document can get created naturally later and
then follow the usual cycle. But, at times, such documents are ephemeral in
nature and never re-used. In those cases, such documents can remain silently
without getting collected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2953) Implement text extractor as part of oak-run
[ https://issues.apache.org/jira/browse/OAK-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627610#comment-14627610 ] Chetan Mehrotra commented on OAK-2953: -- * Added support for generate http://svn.apache.org/r1690636 * Updated readme - http://svn.apache.org/r1691128 Merged the changes to 1.2 and 1.0 branches Implement text extractor as part of oak-run --- Key: OAK-2953 URL: https://issues.apache.org/jira/browse/OAK-2953 Project: Jackrabbit Oak Issue Type: Sub-task Components: run Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Fix For: 1.3.3 Attachments: OAK-2953.patch Implement a crawler and indexer which can find out all binary content in repository under certain path and extracts text from them and store them somewhere -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2892) Speed up lucene indexing post migration by pre extracting the text content from binaries
[ https://issues.apache.org/jira/browse/OAK-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627613#comment-14627613 ] Chetan Mehrotra commented on OAK-2892: -- Documentation updated in http://jackrabbit.apache.org/oak/docs/query/lucene.html#text-extraction Feature merged to 1.0 and 1.2 branches Speed up lucene indexing post migration by pre extracting the text content from binaries Key: OAK-2892 URL: https://issues.apache.org/jira/browse/OAK-2892 Project: Jackrabbit Oak Issue Type: New Feature Components: lucene, run Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Labels: performance Fix For: 1.2.3, 1.3.3, 1.0.18 While migrating large repositories say having 3 M docs (250k PDF) Lucene indexing takes long time to complete (at time 4 days!). Currently the text extraction logic is coupled with Lucene indexing and hence is performed in a single threaded mode which slows down the indexing process. Further if the reindexing has to be triggered it has to be done all over again. To speed up the Lucene indexing we can decouple the text extraction from actual indexing. It is partly based on discussion on OAK-2787 # Introduce a new ExtractedTextProvider which can provide extracted text for a given Blob instance # In oak-run introduce a new indexer mode - This would take a path in repository and would then traverse the repository and look for existing binaries and extract text from that So before or after migration is done one can run this oak-run tool to create this store which has the text already extracted. Then post startup we need to wire up the ExtractedTextProvider instance (which is backed by the BlobStore populated before) and indexing logic can just get content from that. This would avoid performing expensive text extraction in the indexing thread. See discussion thread http://markmail.org/thread/ndlfpkwfgpey6o66 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2892) Speed up lucene indexing post migration by pre extracting the text content from binaries
[ https://issues.apache.org/jira/browse/OAK-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-2892: - Fix Version/s: 1.2.3 Speed up lucene indexing post migration by pre extracting the text content from binaries Key: OAK-2892 URL: https://issues.apache.org/jira/browse/OAK-2892 Project: Jackrabbit Oak Issue Type: New Feature Components: lucene, run Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Labels: performance Fix For: 1.2.3, 1.3.3, 1.0.18 While migrating large repositories say having 3 M docs (250k PDF) Lucene indexing takes long time to complete (at time 4 days!). Currently the text extraction logic is coupled with Lucene indexing and hence is performed in a single threaded mode which slows down the indexing process. Further if the reindexing has to be triggered it has to be done all over again. To speed up the Lucene indexing we can decouple the text extraction from actual indexing. It is partly based on discussion on OAK-2787 # Introduce a new ExtractedTextProvider which can provide extracted text for a given Blob instance # In oak-run introduce a new indexer mode - This would take a path in repository and would then traverse the repository and look for existing binaries and extract text from that So before or after migration is done one can run this oak-run tool to create this store which has the text already extracted. Then post startup we need to wire up the ExtractedTextProvider instance (which is backed by the BlobStore populated before) and indexing logic can just get content from that. This would avoid performing expensive text extraction in the indexing thread. See discussion thread http://markmail.org/thread/ndlfpkwfgpey6o66 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-1842) ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
[
https://issues.apache.org/jira/browse/OAK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627896#comment-14627896
]
Konrad Windszus commented on OAK-1842:
--
Wouldn't it be a good idea to limit the length of the blobIDs being used by the
FileBlobStore to only 4095 bytes, until
https://issues.apache.org/jira/browse/OAK-3107 is fixed?
ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
Key: OAK-1842
URL: https://issues.apache.org/jira/browse/OAK-1842
Project: Jackrabbit Oak
Issue Type: Bug
Components: blob
Affects Versions: 1.0
Reporter: Konrad Windszus
Assignee: Francesco Mari
Labels: resilience
Fix For: 1.3.3
The stacktrace of the call shows something like
{code}
20.05.2014 11:13:07.428 *ERROR* [OsgiInstallerImpl]
com.adobe.granite.installer.factory.packages.impl.PackageTransformer Error
while processing install task.
java.lang.IllegalStateException: Unexpected value record type: f2
at
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.length(SegmentBlob.java:101)
at
org.apache.jackrabbit.oak.plugins.value.BinaryImpl.getSize(BinaryImpl.java:74)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:435)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:376)
at
org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.getPackage(JcrPackageImpl.java:324)
{code}
The blob store was configured correctly and according to the log also
correctly initialized
{code}
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService
Initializing SegmentNodeStore with BlobStore
[org.apache.jackrabbit.oak.spi.blob.FileBlobStore@7e3dec43]
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService Component
still not activated. Ignoring the initialization call
20.05.2014 11:11:07.077 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK opened:
crx-quickstart/repository/segmentstore (mmap=true)
{code}
Under which circumstances can the length within the SegmentBlob be invalid?
This only happens if a File Blob Store is configured
(http://jackrabbit.apache.org/oak/docs/osgi_config.html). If a file datastore
is used, there is no such exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3108) RDBDocumentStore: improve diagnostics for failed inserts/updates caused by long data
[ https://issues.apache.org/jira/browse/OAK-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3108: Summary: RDBDocumentStore: improve diagnostics for failed inserts/updates caused by long data (was: RDBDocumentStore: improve diagnostics for failed inserts/updates causes by long data) RDBDocumentStore: improve diagnostics for failed inserts/updates caused by long data Key: OAK-3108 URL: https://issues.apache.org/jira/browse/OAK-3108 Project: Jackrabbit Oak Issue Type: Sub-task Components: rdbmk Affects Versions: 1.2.2, 1.3.2, 1.0.17 Reporter: Julian Reschke Assignee: Julian Reschke Fix For: 1.3.3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3011) Add name of lucene-property index to cost debug log
[
https://issues.apache.org/jira/browse/OAK-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-3011:
-
Fix Version/s: 1.2.3
Add name of lucene-property index to cost debug log
---
Key: OAK-3011
URL: https://issues.apache.org/jira/browse/OAK-3011
Project: Jackrabbit Oak
Issue Type: Improvement
Components: lucene, query
Reporter: Alex Parvulescu
Assignee: Alex Parvulescu
Fix For: 1.2.3, 1.3.2
Attachments: OAK-3011-v2.patch, OAK-3011.patch
Currently the cost debug log only contains the type and the cost, but if
there are multiple lucene-property indexes, there's no way of knowing which
index has what cost so it would be really nice to have the index name
included with the cost output.
Now:
{code}
org.apache.jackrabbit.oak.query.QueryImpl cost for lucene-property is 1.5
{code}
Nice to have:
{code}
org.apache.jackrabbit.oak.query.QueryImpl cost for lucene-property [name] is
1.5
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-1842) ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
[
https://issues.apache.org/jira/browse/OAK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627894#comment-14627894
]
Francesco Mari commented on OAK-1842:
-
Yes, right. The length of the blob ID generated by the {{FileDataStore}} is way
below the threshold that triggered the bug discussed in OAK-3105.
ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
Key: OAK-1842
URL: https://issues.apache.org/jira/browse/OAK-1842
Project: Jackrabbit Oak
Issue Type: Bug
Components: blob
Affects Versions: 1.0
Reporter: Konrad Windszus
Assignee: Francesco Mari
Labels: resilience
Fix For: 1.3.3
The stacktrace of the call shows something like
{code}
20.05.2014 11:13:07.428 *ERROR* [OsgiInstallerImpl]
com.adobe.granite.installer.factory.packages.impl.PackageTransformer Error
while processing install task.
java.lang.IllegalStateException: Unexpected value record type: f2
at
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.length(SegmentBlob.java:101)
at
org.apache.jackrabbit.oak.plugins.value.BinaryImpl.getSize(BinaryImpl.java:74)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:435)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:376)
at
org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.getPackage(JcrPackageImpl.java:324)
{code}
The blob store was configured correctly and according to the log also
correctly initialized
{code}
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService
Initializing SegmentNodeStore with BlobStore
[org.apache.jackrabbit.oak.spi.blob.FileBlobStore@7e3dec43]
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService Component
still not activated. Ignoring the initialization call
20.05.2014 11:11:07.077 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK opened:
crx-quickstart/repository/segmentstore (mmap=true)
{code}
Under which circumstances can the length within the SegmentBlob be invalid?
This only happens if a File Blob Store is configured
(http://jackrabbit.apache.org/oak/docs/osgi_config.html). If a file datastore
is used, there is no such exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-3098) CopyOnWrite might block Async indexer thread indefinitely
[
https://issues.apache.org/jira/browse/OAK-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra resolved OAK-3098.
--
Resolution: Fixed
Fix Version/s: (was: 1.2.4)
1.2.3
Fixed by ensuring that when oak-lucene bundle gets restarted it closes the
IndexCopier and which in turn prevent any further task from being added to
queue. Further the latch is now awaited with a timeout which should further
prevent any such occurance
* trunk - http://svn.apache.org/r1691167
* 1.0 - http://svn.apache.org/r1691168
* 1.2 - http://svn.apache.org/r1691172
CopyOnWrite might block Async indexer thread indefinitely
-
Key: OAK-3098
URL: https://issues.apache.org/jira/browse/OAK-3098
Project: Jackrabbit Oak
Issue Type: Bug
Components: lucene
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Priority: Minor
Fix For: 1.2.3, 1.3.3, 1.0.18
CopyOnWrite logic uses a CountDownLatch to sync up on close call. In case of
config change which might lead to closing of the the IndexCopier this latch
might not be released causing the async index thread to hang
{noformat}
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
org.apache.jackrabbit.oak.plugins.index.lucene.IndexCopier$CopyOnWriteDirectory.close(IndexCopier.java:648)
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditorContext.closeWriter(LuceneIndexEditorContext.java:196)
org.apache.jackrabbit.oak.plugins.index.lucene.LuceneIndexEditor.leave(LuceneIndexEditor.java:207)
org.apache.jackrabbit.oak.plugins.index.IndexUpdate.leave(IndexUpdate.java:219)
org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:56)
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate.updateIndex(AsyncIndexUpdate.java:366)
org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate.run(AsyncIndexUpdate.java:311)
org.apache.sling.commons.scheduler.impl.QuartzJobExecutor.execute(QuartzJobExecutor.java:105)
org.quartz.core.JobRunShell.run(JobRunShell.java:207)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)
{noformat}
Ideally this latch should have a timeout or closing logic of IndexCopier
ensures that this is released
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3103) Stale document in MongoDocumentStore cache
[
https://issues.apache.org/jira/browse/OAK-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627955#comment-14627955
]
Marcel Reutegger commented on OAK-3103:
---
Fix implemented in trunk and test enabled: http://svn.apache.org/r1691188
I renamed {{applyToCache()}} to {{updateCache()}} and simplified it. The method
now only deals with cache updates when a caller has a previously cached
document ({{oldDoc}} must be non-null now).
[~chetanm], can you please review?
Stale document in MongoDocumentStore cache
--
Key: OAK-3103
URL: https://issues.apache.org/jira/browse/OAK-3103
Project: Jackrabbit Oak
Issue Type: Bug
Components: core, mongomk
Affects Versions: 1.2.2, 1.3.2, 1.0.17
Reporter: Marcel Reutegger
Assignee: Marcel Reutegger
In some rare cases it may happen that a stale document is put into the
MongoDocumentStore cache.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-3105) SegmentWriter doesn't properly check the length of external blob IDs
[
https://issues.apache.org/jira/browse/OAK-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig reassigned OAK-3105:
--
Assignee: Michael Dürig
SegmentWriter doesn't properly check the length of external blob IDs
Key: OAK-3105
URL: https://issues.apache.org/jira/browse/OAK-3105
Project: Jackrabbit Oak
Issue Type: Bug
Affects Versions: 1.2.2, 1.3.2, 1.0.17
Reporter: Francesco Mari
Assignee: Michael Dürig
Priority: Critical
Attachments: OAK-3105-01.patch
To store the length field of an external binary ID, the following encoding is
used:
{noformat}
1110 + 4bit + 8bit
{noformat}
which allows to store numbers between 0 and 2{^}12^ - 1.
The current implementation of {{SegmentWriter}} allows the length of binary
IDs to range between 0 and 2{^}13^ - 1, writing incorrect data when the
length of the binary ID ranges from 2{^}12^ to 2{^}13^ - 1.
When reading this incorrect data back, an {{IllegalStateException}} is thrown
complaining that the first byte of the length fields has an unexpected value
record type. See OAK-1842 for an example.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3011) Add name of lucene-property index to cost debug log
[
https://issues.apache.org/jira/browse/OAK-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627985#comment-14627985
]
Alex Parvulescu commented on OAK-3011:
--
merged to 1.2 branch (r 1691197), except for the
src/main/java/org/apache/jackrabbit/oak/spi/query/package-info.java file which
contained a major increase of the OSGi export version which would break all
upstream dependencies. this change came in via OAK-2926 so there was no way
around it.
Add name of lucene-property index to cost debug log
---
Key: OAK-3011
URL: https://issues.apache.org/jira/browse/OAK-3011
Project: Jackrabbit Oak
Issue Type: Improvement
Components: lucene, query
Reporter: Alex Parvulescu
Assignee: Alex Parvulescu
Fix For: 1.2.3, 1.3.2
Attachments: OAK-3011-v2.patch, OAK-3011.patch
Currently the cost debug log only contains the type and the cost, but if
there are multiple lucene-property indexes, there's no way of knowing which
index has what cost so it would be really nice to have the index name
included with the cost output.
Now:
{code}
org.apache.jackrabbit.oak.query.QueryImpl cost for lucene-property is 1.5
{code}
Nice to have:
{code}
org.apache.jackrabbit.oak.query.QueryImpl cost for lucene-property [name] is
1.5
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3109) OOME in tarkmk standby tests
[
https://issues.apache.org/jira/browse/OAK-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627986#comment-14627986
]
Michael Dürig commented on OAK-3109:
Many of the {{SegmentTracker}} instances are referenced from
{{SegmentLoaderHandler}}, which is registered to the channel pipeline as a
channel handler. I don't know who would be responsible to eventually clean this
up. However explicitly removing the handler again in {{close()}} fixes the OOME:
{code}
---
oak-tarmk-standby/src/main/java/org/apache/jackrabbit/oak/plugins/segment/standby/client/StandbyClientHandler.java
(date 1436962414000)
+++
oak-tarmk-standby/src/main/java/org/apache/jackrabbit/oak/plugins/segment/standby/client/StandbyClientHandler.java
(revision )
@@ -82,6 +82,7 @@
ctx.flush();
}
+SegmentLoaderHandler h2;
synchronized void setHead(RecordId head) {
if (store.getHead().getRecordId().equals(head)) {
@@ -97,7 +98,7 @@
ctx.pipeline().addLast(new ReplyDecoder(store));
loaderExecutor = new DefaultEventExecutorGroup(4);
-SegmentLoaderHandler h2 = new SegmentLoaderHandler(store, head,
+h2 = new SegmentLoaderHandler(store, head,
loaderExecutor, this.observer.getID(), running, readTimeoutMs,
autoClean);
ctx.pipeline().addLast(loaderExecutor, h2);
@@ -115,6 +116,10 @@
@Override
public void close() {
+if (h2 != null) {
+ctx.pipeline().remove(h2);
+h2 = null;
+}
ctx.close();
if (!executor.isShuttingDown()) {
executor.shutdownGracefully(1, 2, TimeUnit.SECONDS)
{code}
OOME in tarkmk standby tests
Key: OAK-3109
URL: https://issues.apache.org/jira/browse/OAK-3109
Project: Jackrabbit Oak
Issue Type: Bug
Components: tarmk-standby
Reporter: Michael Dürig
Running the tarmk-standby ITs results in an OOME when running with the new
string cache from OAK-3007. Looking at the heap dump I see many
{{SegmentTracker}} instances around, which leads me to think that the test
setup might have a leak somewhere.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-3108) RDBDocumentStore: improve diagnostics for failed inserts/updates causes by long data
[ https://issues.apache.org/jira/browse/OAK-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke reassigned OAK-3108: --- Assignee: Julian Reschke RDBDocumentStore: improve diagnostics for failed inserts/updates causes by long data Key: OAK-3108 URL: https://issues.apache.org/jira/browse/OAK-3108 Project: Jackrabbit Oak Issue Type: Sub-task Components: rdbmk Affects Versions: 1.2.2, 1.3.2, 1.0.17 Reporter: Julian Reschke Assignee: Julian Reschke -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3108) RDBDocumentStore: improve diagnostics for failed inserts/updates causes by long data
Julian Reschke created OAK-3108: --- Summary: RDBDocumentStore: improve diagnostics for failed inserts/updates causes by long data Key: OAK-3108 URL: https://issues.apache.org/jira/browse/OAK-3108 Project: Jackrabbit Oak Issue Type: Sub-task Components: rdbmk Affects Versions: 1.0.17, 1.3.2, 1.2.2 Reporter: Julian Reschke -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-1842) ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
[
https://issues.apache.org/jira/browse/OAK-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627903#comment-14627903
]
Francesco Mari commented on OAK-1842:
-
The {{BlobStore}} API doesn't impose any limitation on the size of the
{{String}} returned as a blob ID. I think it's a good thing to leave freedom to
the {{BlobStore}} implementation, because sometimes it may be more performant
to return a large blob ID instead of interacting with the underlying storage
layer (the file system, in the case of {{FileBlobStore}}).
I think that {{SegmentWriter}} should be ready to accept any blob ID,
independently from the length. An implementation of {{BlobStore}} that returns
very large strings shouldn't be allowed to break the internal representation of
the segment.
ISE: Unexpected value record type: f2 is thrown when FileBlobStore is used
Key: OAK-1842
URL: https://issues.apache.org/jira/browse/OAK-1842
Project: Jackrabbit Oak
Issue Type: Bug
Components: blob
Affects Versions: 1.0
Reporter: Konrad Windszus
Assignee: Francesco Mari
Labels: resilience
Fix For: 1.3.3
The stacktrace of the call shows something like
{code}
20.05.2014 11:13:07.428 *ERROR* [OsgiInstallerImpl]
com.adobe.granite.installer.factory.packages.impl.PackageTransformer Error
while processing install task.
java.lang.IllegalStateException: Unexpected value record type: f2
at
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.length(SegmentBlob.java:101)
at
org.apache.jackrabbit.oak.plugins.value.BinaryImpl.getSize(BinaryImpl.java:74)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:435)
at
org.apache.jackrabbit.oak.jcr.session.PropertyImpl.getLength(PropertyImpl.java:376)
at
org.apache.jackrabbit.vault.packaging.impl.JcrPackageImpl.getPackage(JcrPackageImpl.java:324)
{code}
The blob store was configured correctly and according to the log also
correctly initialized
{code}
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService
Initializing SegmentNodeStore with BlobStore
[org.apache.jackrabbit.oak.spi.blob.FileBlobStore@7e3dec43]
20.05.2014 11:11:07.029 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService Component
still not activated. Ignoring the initialization call
20.05.2014 11:11:07.077 *INFO* [FelixStartLevel]
org.apache.jackrabbit.oak.plugins.segment.file.FileStore TarMK opened:
crx-quickstart/repository/segmentstore (mmap=true)
{code}
Under which circumstances can the length within the SegmentBlob be invalid?
This only happens if a File Blob Store is configured
(http://jackrabbit.apache.org/oak/docs/osgi_config.html). If a file datastore
is used, there is no such exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3099) Revision GC fails when split documents with very long paths are present
[
https://issues.apache.org/jira/browse/OAK-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amit Jain updated OAK-3099:
---
Attachment: OAK-3099.patch
[~mreutegg], [~chetanm]
Could you please review the patch which incorporates the test and fix provided
by [~Csaba Varga].
Revision GC fails when split documents with very long paths are present
---
Key: OAK-3099
URL: https://issues.apache.org/jira/browse/OAK-3099
Project: Jackrabbit Oak
Issue Type: Bug
Components: mongomk
Affects Versions: 1.0.13
Reporter: Csaba Varga
Priority: Minor
Attachments: OAK-3099.patch, SplitDocumentGenerator.java
My company is using the MongoDB microkernel with Oak, and we've noticed that
the daily revision GC is failing with errors like this:
{code}
13.07.2015 13:06:16.261 *ERROR* [pool-7-thread-1-Maintenance
Queue(com/adobe/granite/maintenance/job/RevisionCleanupTask)]
org.apache.jackrabbit.oak.management.ManagementOperation Revision garbage
collection failed
java.lang.IllegalArgumentException:
13:h113f9d0fe7ac0f87fa06397c37b9ffd4b372eeb1ec93e0818bb4024a32587820
at
org.apache.jackrabbit.oak.plugins.document.Revision.fromString(Revision.java:236)
at
org.apache.jackrabbit.oak.plugins.document.SplitDocumentCleanUp.disconnect(SplitDocumentCleanUp.java:84)
at
org.apache.jackrabbit.oak.plugins.document.SplitDocumentCleanUp.disconnect(SplitDocumentCleanUp.java:56)
at
org.apache.jackrabbit.oak.plugins.document.VersionGCSupport.deleteSplitDocuments(VersionGCSupport.java:53)
at
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector.collectSplitDocuments(VersionGarbageCollector.java:117)
at
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollector.gc(VersionGarbageCollector.java:105)
at
org.apache.jackrabbit.oak.plugins.document.DocumentNodeStoreService$2.run(DocumentNodeStoreService.java:511)
at org.apache.jackrabbit.oak.spi.state.RevisionGC$1.call(RevisionGC.java:68)
at org.apache.jackrabbit.oak.spi.state.RevisionGC$1.call(RevisionGC.java:64)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}
I've narrowed the issue down to the disconnect(NodeDocument) method of the
[SplitDocumentCleanUp
class|https://svn.apache.org/repos/asf/jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/document/SplitDocumentCleanUp.java].
The method always tries to extract the path of the node from its ID, but
this won't work for documents whose path is very long because those documents
will have the hash of their path in the ID.
I believe this code should fix the issue, but I haven't had a chance to
actually try it:
{code}
private void disconnect(NodeDocument splitDoc) {
String mainId = Utils.getIdFromPath(splitDoc.getMainPath());
NodeDocument doc = store.find(NODES, mainId);
if (doc == null) {
LOG.warn(Main document {} already removed. Split document is {},
mainId, splitId);
return;
}
String path = splitDoc.getPath();
int slashIdx = path.lastIndexOf('/');
int height = Integer.parseInt(path.substring(slashIdx + 1));
Revision rev = Revision.fromString(
path.substring(path.lastIndexOf('/', slashIdx - 1) + 1,
slashIdx));
doc = doc.findPrevReferencingDoc(rev, height);
if (doc == null) {
LOG.warn(Split document {} not referenced anymore. Main document
is {},
splitId, mainId);
return;
}
// remove reference
if (doc.getSplitDocType() == INTERMEDIATE) {
disconnectFromIntermediate(doc, rev);
} else {
markStaleOnMain(doc, rev, height);
}
}
{code}
By using getPath(), the code should automatically use either the ID or the
_path property, whichever is right for the document.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3108) RDBDocumentStore: improve diagnostics for failed inserts/updates causes by long data
[ https://issues.apache.org/jira/browse/OAK-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke updated OAK-3108: Fix Version/s: 1.3.3 RDBDocumentStore: improve diagnostics for failed inserts/updates causes by long data Key: OAK-3108 URL: https://issues.apache.org/jira/browse/OAK-3108 Project: Jackrabbit Oak Issue Type: Sub-task Components: rdbmk Affects Versions: 1.2.2, 1.3.2, 1.0.17 Reporter: Julian Reschke Assignee: Julian Reschke Fix For: 1.3.3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3109) OOME in tarkmk standby tests
Michael Dürig created OAK-3109:
--
Summary: OOME in tarkmk standby tests
Key: OAK-3109
URL: https://issues.apache.org/jira/browse/OAK-3109
Project: Jackrabbit Oak
Issue Type: Bug
Components: tarmk-standby
Reporter: Michael Dürig
Running the tarmk-standby ITs results in an OOME when running with the new
string cache from OAK-3007. Looking at the heap dump I see many
{{SegmentTracker}} instances around, which leads me to think that the test
setup might have a leak somewhere.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2619) Repeated upgrades
[ https://issues.apache.org/jira/browse/OAK-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manfred Baedke updated OAK-2619: Fix Version/s: 1.3.3 Repeated upgrades - Key: OAK-2619 URL: https://issues.apache.org/jira/browse/OAK-2619 Project: Jackrabbit Oak Issue Type: New Feature Components: upgrade Affects Versions: 1.1.7 Reporter: Julian Sedding Assignee: Manfred Baedke Priority: Minor Fix For: 1.3.3 Attachments: OAK-2619-v2.patch, OAK-2619.patch, incremental-upgrade-no-changes-mongo.png, incremental-upgrade-no-changes-tar.png, initial-upgrade-mongo.png, initial-upgrade-tar.png When upgrading from Jackrabbit 2 to Oak there are several scenarios that could benefit from the ability to upgrade repeatedly into one target repository. E.g. a migration process might look as follows: # upgrade a backup of a large repository a week before go-live # run the upgrade again every night (commit-hooks only handle delta) # run the upgrade one final time before go-live (commit-hooks only handle delta) In this scenario each upgrade would require a full traversal of the source repository. However, if done right, only the delta needs to be written and the commit-hooks also only need to process the delta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2619) Repeated upgrades
[ https://issues.apache.org/jira/browse/OAK-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628802#comment-14628802 ] Manfred Baedke commented on OAK-2619: - Committed to trunk with revision 1691280. [~jsedding], besides trivial modifications I re-added the check for the maximal length of names. If you removed it on purpose and I didn't get it, please explain. Repeated upgrades - Key: OAK-2619 URL: https://issues.apache.org/jira/browse/OAK-2619 Project: Jackrabbit Oak Issue Type: New Feature Components: upgrade Affects Versions: 1.1.7 Reporter: Julian Sedding Assignee: Manfred Baedke Priority: Minor Fix For: 1.3.3 Attachments: OAK-2619-v2.patch, OAK-2619.patch, incremental-upgrade-no-changes-mongo.png, incremental-upgrade-no-changes-tar.png, initial-upgrade-mongo.png, initial-upgrade-tar.png When upgrading from Jackrabbit 2 to Oak there are several scenarios that could benefit from the ability to upgrade repeatedly into one target repository. E.g. a migration process might look as follows: # upgrade a backup of a large repository a week before go-live # run the upgrade again every night (commit-hooks only handle delta) # run the upgrade one final time before go-live (commit-hooks only handle delta) In this scenario each upgrade would require a full traversal of the source repository. However, if done right, only the delta needs to be written and the commit-hooks also only need to process the delta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2619) Repeated upgrades
[ https://issues.apache.org/jira/browse/OAK-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manfred Baedke resolved OAK-2619. - Resolution: Fixed Repeated upgrades - Key: OAK-2619 URL: https://issues.apache.org/jira/browse/OAK-2619 Project: Jackrabbit Oak Issue Type: New Feature Components: upgrade Affects Versions: 1.1.7 Reporter: Julian Sedding Assignee: Manfred Baedke Priority: Minor Fix For: 1.3.3 Attachments: OAK-2619-v2.patch, OAK-2619.patch, incremental-upgrade-no-changes-mongo.png, incremental-upgrade-no-changes-tar.png, initial-upgrade-mongo.png, initial-upgrade-tar.png When upgrading from Jackrabbit 2 to Oak there are several scenarios that could benefit from the ability to upgrade repeatedly into one target repository. E.g. a migration process might look as follows: # upgrade a backup of a large repository a week before go-live # run the upgrade again every night (commit-hooks only handle delta) # run the upgrade one final time before go-live (commit-hooks only handle delta) In this scenario each upgrade would require a full traversal of the source repository. However, if done right, only the delta needs to be written and the commit-hooks also only need to process the delta. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-3055) Improve segment cache in SegmentTracker
[
https://issues.apache.org/jira/browse/OAK-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-3055.
Resolution: Fixed
Fix Version/s: (was: 1.3.5)
1.3.3
Replaced the segment cache with a LIRS implementation at
http://svn.apache.org/r1691219. In addition http://svn.apache.org/r1691220
exposes the {{CacheStatsMBean}} for this new cache.
I can confirm that this in conjunction with OAK-3007 considerably reduces the
observer lock contention.
Improve segment cache in SegmentTracker
---
Key: OAK-3055
URL: https://issues.apache.org/jira/browse/OAK-3055
Project: Jackrabbit Oak
Issue Type: Improvement
Components: segmentmk
Reporter: Michael Dürig
Assignee: Michael Dürig
Labels: resilience, scalability
Fix For: 1.3.3
Attachments: OAK-3055.patch
The hand crafted segment cache in {{SegmentTracker}} is prone to lock
contentions in concurrent access scenarios. As {{SegmentNodeStore#merge}}
might also end up acquiring this lock while holding the commit semaphore the
situation can easily lead to many threads being blocked on the commit
semaphore. The {{SegmentTracker}} cache doesn't differentiate between read
and write access, which means that reader threads can block writer threads.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3007) SegmentStore cache does not take string map into account
[ https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-3007: --- Labels: doc-impacting (was: ) SegmentStore cache does not take string map into account -- Key: OAK-3007 URL: https://issues.apache.org/jira/browse/OAK-3007 Project: Jackrabbit Oak Issue Type: Bug Components: segmentmk Reporter: Thomas Mueller Assignee: Michael Dürig Labels: doc-impacting Fix For: 1.3.3 Attachments: OAK-3007-2.patch, OAK-3007-3.patch, OAK-3007.patch The SegmentStore cache size calculation ignores the size of the field Segment.string (a concurrent hash map). It looks like a regular segment in a memory mapped file has the size 1024, no matter how many strings are loaded in memory. This can lead to out of memory. There seems to be no way to limit (configure) the amount of memory used by strings. In one example, 100'000 segments are loaded in memory, and 5 GB are used for Strings in that map. We need a way to configure the amount of memory used for that. This seems to be basically a cache. OAK-2688 does this, but it would be better to have one cache with a configurable size limit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3055) Improve segment cache in SegmentTracker
[
https://issues.apache.org/jira/browse/OAK-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628126#comment-14628126
]
Michael Dürig commented on OAK-3055:
I went with an implementation along the lines of this \[1]. Although in my case
I *do have* concurrent threads accessing the rng, I figured this is not too
problematic as the race in updating {{x}} will only create a slight tilt in the
random numbers, which should not be problematic for the given use case.
An alternative implementation could use {{AtomicInteger.compareAndSet}} for
{{x}}:
{code}
int i = x.get();
int j = 0xc3e157c1 - rotateLeft(i, 19);
while (!x.compareAndSet(i, j)) {
i = x.get();
j = 0xc3e157c1 - rotateLeft(i, 19);
}
{code}
Please go ahead if you think this could be further optimised.
\[1]
http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/segment/SegmentId.java?r1=1691220r2=1691219pathrev=1691220
Improve segment cache in SegmentTracker
---
Key: OAK-3055
URL: https://issues.apache.org/jira/browse/OAK-3055
Project: Jackrabbit Oak
Issue Type: Improvement
Components: segmentmk
Reporter: Michael Dürig
Assignee: Michael Dürig
Labels: doc-impacting, resilience, scalability
Fix For: 1.3.3
Attachments: OAK-3055.patch
The hand crafted segment cache in {{SegmentTracker}} is prone to lock
contentions in concurrent access scenarios. As {{SegmentNodeStore#merge}}
might also end up acquiring this lock while holding the commit semaphore the
situation can easily lead to many threads being blocked on the commit
semaphore. The {{SegmentTracker}} cache doesn't differentiate between read
and write access, which means that reader threads can block writer threads.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3055) Improve segment cache in SegmentTracker
[
https://issues.apache.org/jira/browse/OAK-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-3055:
---
Labels: doc-impacting resilience scalability (was: resilience scalability)
Improve segment cache in SegmentTracker
---
Key: OAK-3055
URL: https://issues.apache.org/jira/browse/OAK-3055
Project: Jackrabbit Oak
Issue Type: Improvement
Components: segmentmk
Reporter: Michael Dürig
Assignee: Michael Dürig
Labels: doc-impacting, resilience, scalability
Fix For: 1.3.3
Attachments: OAK-3055.patch
The hand crafted segment cache in {{SegmentTracker}} is prone to lock
contentions in concurrent access scenarios. As {{SegmentNodeStore#merge}}
might also end up acquiring this lock while holding the commit semaphore the
situation can easily lead to many threads being blocked on the commit
semaphore. The {{SegmentTracker}} cache doesn't differentiate between read
and write access, which means that reader threads can block writer threads.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3007) SegmentStore cache does not take string map into account
[ https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-3007: --- Labels: doc-impacting resilience scalability (was: doc-impacting) SegmentStore cache does not take string map into account -- Key: OAK-3007 URL: https://issues.apache.org/jira/browse/OAK-3007 Project: Jackrabbit Oak Issue Type: Bug Components: segmentmk Reporter: Thomas Mueller Assignee: Michael Dürig Labels: doc-impacting, resilience, scalability Fix For: 1.3.3 Attachments: OAK-3007-2.patch, OAK-3007-3.patch, OAK-3007.patch The SegmentStore cache size calculation ignores the size of the field Segment.string (a concurrent hash map). It looks like a regular segment in a memory mapped file has the size 1024, no matter how many strings are loaded in memory. This can lead to out of memory. There seems to be no way to limit (configure) the amount of memory used by strings. In one example, 100'000 segments are loaded in memory, and 5 GB are used for Strings in that map. We need a way to configure the amount of memory used for that. This seems to be basically a cache. OAK-2688 does this, but it would be better to have one cache with a configurable size limit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2953) Implement text extractor as part of oak-run
[ https://issues.apache.org/jira/browse/OAK-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-2953. -- Resolution: Fixed Fix Version/s: 1.0.18 1.2.3 Implement text extractor as part of oak-run --- Key: OAK-2953 URL: https://issues.apache.org/jira/browse/OAK-2953 Project: Jackrabbit Oak Issue Type: Sub-task Components: run Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Fix For: 1.2.3, 1.3.3, 1.0.18 Attachments: OAK-2953.patch Implement a crawler and indexer which can find out all binary content in repository under certain path and extracts text from them and store them somewhere -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2989) Swap large commits to disk in order to avoid OOME
[ https://issues.apache.org/jira/browse/OAK-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-2989: --- Fix Version/s: (was: 1.3.3) Swap large commits to disk in order to avoid OOME - Key: OAK-2989 URL: https://issues.apache.org/jira/browse/OAK-2989 Project: Jackrabbit Oak Issue Type: Bug Components: core Affects Versions: 1.2.2 Reporter: Timothee Maret As described in [0] large commits consume a fair amount of memory. With very large commits, this become problematic as commits may eat up 100GB or more and thus causing OOME and aborting the commit. Instead of keeping the whole commit in memory, the implementation may store parts of it on the disk once the heap memory consumption reaches a configurable threshold. This would allow to solve the issue and not simply mitigate it as in OAK-2968, OAK-2969. The behaviour may already be supported for some configurations of Oak. At least the setup Mongo + DocumentStore seemed not to support it. [0] http://permalink.gmane.org/gmane.comp.apache.jackrabbit.oak.devel/8196 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2989) Swap large commits to disk in order to avoid OOME
[ https://issues.apache.org/jira/browse/OAK-2989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628274#comment-14628274 ] Michael Dürig commented on OAK-2989: Unscheduling as this issue is not fully qualified yet: is it a regressions or is this a symptom of another issue? Swap large commits to disk in order to avoid OOME - Key: OAK-2989 URL: https://issues.apache.org/jira/browse/OAK-2989 Project: Jackrabbit Oak Issue Type: Bug Components: core Affects Versions: 1.2.2 Reporter: Timothee Maret As described in [0] large commits consume a fair amount of memory. With very large commits, this become problematic as commits may eat up 100GB or more and thus causing OOME and aborting the commit. Instead of keeping the whole commit in memory, the implementation may store parts of it on the disk once the heap memory consumption reaches a configurable threshold. This would allow to solve the issue and not simply mitigate it as in OAK-2968, OAK-2969. The behaviour may already be supported for some configurations of Oak. At least the setup Mongo + DocumentStore seemed not to support it. [0] http://permalink.gmane.org/gmane.comp.apache.jackrabbit.oak.devel/8196 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2795) TarMK revision cleanup message too verbose.
[
https://issues.apache.org/jira/browse/OAK-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-2795:
---
Fix Version/s: (was: 1.3.3)
TarMK revision cleanup message too verbose.
---
Key: OAK-2795
URL: https://issues.apache.org/jira/browse/OAK-2795
Project: Jackrabbit Oak
Issue Type: Improvement
Components: segmentmk
Reporter: Zygmunt Wiercioch
Priority: Minor
The segment GC message can be thousands of lines long on a system which
experienced a lot of activity. For example, in my test, I had:
{noformat}
org.apache.jackrabbit.oak.plugins.segment.file.TarReader-GC Cleaned segments
from data8a.tar:
4b6882e7-babe-4854-a966-c3a630c338c6, 506a2eb3-92a6-4b83-a320-5cc5ac304302,
f179e4b1-acec-4a5d-a686-bf5e97828563, 8c6b3a4d-c327-4701-affb-fcd055c4aa67,
{noformat}
followed by approximately 40,000 more lines.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2795) TarMK revision cleanup message too verbose.
[
https://issues.apache.org/jira/browse/OAK-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628276#comment-14628276
]
Michael Dürig commented on OAK-2795:
Unscheduling as for now we need those message. The suggestion is to configure
logging such that those messages are not logged, which will adversely impact
post mortem analysis in the case of an SNFE.
TarMK revision cleanup message too verbose.
---
Key: OAK-2795
URL: https://issues.apache.org/jira/browse/OAK-2795
Project: Jackrabbit Oak
Issue Type: Improvement
Components: segmentmk
Reporter: Zygmunt Wiercioch
Priority: Minor
Labels: gc
The segment GC message can be thousands of lines long on a system which
experienced a lot of activity. For example, in my test, I had:
{noformat}
org.apache.jackrabbit.oak.plugins.segment.file.TarReader-GC Cleaned segments
from data8a.tar:
4b6882e7-babe-4854-a966-c3a630c338c6, 506a2eb3-92a6-4b83-a320-5cc5ac304302,
f179e4b1-acec-4a5d-a686-bf5e97828563, 8c6b3a4d-c327-4701-affb-fcd055c4aa67,
{noformat}
followed by approximately 40,000 more lines.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2795) TarMK revision cleanup message too verbose.
[
https://issues.apache.org/jira/browse/OAK-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-2795:
---
Labels: gc (was: )
TarMK revision cleanup message too verbose.
---
Key: OAK-2795
URL: https://issues.apache.org/jira/browse/OAK-2795
Project: Jackrabbit Oak
Issue Type: Improvement
Components: segmentmk
Reporter: Zygmunt Wiercioch
Priority: Minor
Labels: gc
The segment GC message can be thousands of lines long on a system which
experienced a lot of activity. For example, in my test, I had:
{noformat}
org.apache.jackrabbit.oak.plugins.segment.file.TarReader-GC Cleaned segments
from data8a.tar:
4b6882e7-babe-4854-a966-c3a630c338c6, 506a2eb3-92a6-4b83-a320-5cc5ac304302,
f179e4b1-acec-4a5d-a686-bf5e97828563, 8c6b3a4d-c327-4701-affb-fcd055c4aa67,
{noformat}
followed by approximately 40,000 more lines.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3109) OOME in tarkmk standby tests
[
https://issues.apache.org/jira/browse/OAK-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628007#comment-14628007
]
Michael Dürig commented on OAK-3109:
As an intermediate measure I propose to lower cache sizes for running the
tests. Current cache size is 256MB, which is fine for production but insane for
testing.
OOME in tarkmk standby tests
Key: OAK-3109
URL: https://issues.apache.org/jira/browse/OAK-3109
Project: Jackrabbit Oak
Issue Type: Bug
Components: tarmk-standby
Reporter: Michael Dürig
Running the tarmk-standby ITs results in an OOME when running with the new
string cache from OAK-3007. Looking at the heap dump I see many
{{SegmentTracker}} instances around, which leads me to think that the test
setup might have a leak somewhere.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3109) OOME in tarkmk standby tests
[
https://issues.apache.org/jira/browse/OAK-3109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628009#comment-14628009
]
Michael Dürig commented on OAK-3109:
Changed cache setting at http://svn.apache.org/r1691201
OOME in tarkmk standby tests
Key: OAK-3109
URL: https://issues.apache.org/jira/browse/OAK-3109
Project: Jackrabbit Oak
Issue Type: Bug
Components: tarmk-standby
Reporter: Michael Dürig
Running the tarmk-standby ITs results in an OOME when running with the new
string cache from OAK-3007. Looking at the heap dump I see many
{{SegmentTracker}} instances around, which leads me to think that the test
setup might have a leak somewhere.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2979) JournalTest.cleanupTest reliably fails on Windows
[
https://issues.apache.org/jira/browse/OAK-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julian Reschke updated OAK-2979:
Affects Version/s: 1.2.2
JournalTest.cleanupTest reliably fails on Windows
-
Key: OAK-2979
URL: https://issues.apache.org/jira/browse/OAK-2979
Project: Jackrabbit Oak
Issue Type: Bug
Affects Versions: 1.2.2
Reporter: Julian Reschke
Assignee: Julian Reschke
Attachments: OAK-2979.patch
{noformat}
cleanupTest(org.apache.jackrabbit.oak.plugins.document.JournalTest) Time
elapsed: 0.12 sec FAILURE!
java.lang.AssertionError: expected:0 but was:1
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at
org.apache.jackrabbit.oak.plugins.document.JournalTest.cleanupTest(JournalTest.java:199)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-3108) RDBDocumentStore: improve diagnostics for failed inserts/updates caused by long data
[ https://issues.apache.org/jira/browse/OAK-3108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Reschke resolved OAK-3108. - Resolution: Fixed RDBDocumentStore: improve diagnostics for failed inserts/updates caused by long data Key: OAK-3108 URL: https://issues.apache.org/jira/browse/OAK-3108 Project: Jackrabbit Oak Issue Type: Sub-task Components: rdbmk Affects Versions: 1.2.2, 1.3.2, 1.0.17 Reporter: Julian Reschke Assignee: Julian Reschke Fix For: 1.2.3, 1.3.3, 1.0.18 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-3105) SegmentWriter doesn't properly check the length of external blob IDs
[
https://issues.apache.org/jira/browse/OAK-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig resolved OAK-3105.
Resolution: Fixed
Fix Version/s: 1.0.18
1.3.3
1.2.3
Fixed in trunk at http://svn.apache.org/r1691210, 1.2 at
http://svn.apache.org/r1691214, and 1.0 at http://svn.apache.org/r1691212.
Thanks [~frm] for the patch!
SegmentWriter doesn't properly check the length of external blob IDs
Key: OAK-3105
URL: https://issues.apache.org/jira/browse/OAK-3105
Project: Jackrabbit Oak
Issue Type: Bug
Affects Versions: 1.2.2, 1.3.2, 1.0.17
Reporter: Francesco Mari
Assignee: Michael Dürig
Priority: Critical
Fix For: 1.2.3, 1.3.3, 1.0.18
Attachments: OAK-3105-01.patch
To store the length field of an external binary ID, the following encoding is
used:
{noformat}
1110 + 4bit + 8bit
{noformat}
which allows to store numbers between 0 and 2{^}12^ - 1.
The current implementation of {{SegmentWriter}} allows the length of binary
IDs to range between 0 and 2{^}13^ - 1, writing incorrect data when the
length of the binary ID ranges from 2{^}12^ to 2{^}13^ - 1.
When reading this incorrect data back, an {{IllegalStateException}} is thrown
complaining that the first byte of the length fields has an unexpected value
record type. See OAK-1842 for an example.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-2707) Performance: Consider cache for SessionImpl#getNamespacePrefixes
[ https://issues.apache.org/jira/browse/OAK-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Dürig updated OAK-2707: --- Fix Version/s: (was: 1.3.3) Performance: Consider cache for SessionImpl#getNamespacePrefixes Key: OAK-2707 URL: https://issues.apache.org/jira/browse/OAK-2707 Project: Jackrabbit Oak Issue Type: Improvement Components: jcr Affects Versions: 1.1.7 Reporter: Joel Richard Priority: Critical Labels: performance Attachments: OAK-2707.patch, Screen Shot 2015-03-30 at 08.38.58.png Sesison#getNamespacePrefixes is heavily used in Sling (see org.apache.sling.jcr.resource.JcrPropertyMap#escapeKeyName) and can easily be called 5000-8000 times per page load. Therefore, it adds 10-15% page rendering overhead for Sling-based websites (see attachment). Would it be possible to add either a global cache or at least a session cache for this method? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2707) Performance: Consider cache for SessionImpl#getNamespacePrefixes
[ https://issues.apache.org/jira/browse/OAK-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628253#comment-14628253 ] Michael Dürig commented on OAK-2707: Unscheduling until we have a benchmark. Performance: Consider cache for SessionImpl#getNamespacePrefixes Key: OAK-2707 URL: https://issues.apache.org/jira/browse/OAK-2707 Project: Jackrabbit Oak Issue Type: Improvement Components: jcr Affects Versions: 1.1.7 Reporter: Joel Richard Priority: Critical Labels: performance Attachments: OAK-2707.patch, Screen Shot 2015-03-30 at 08.38.58.png Sesison#getNamespacePrefixes is heavily used in Sling (see org.apache.sling.jcr.resource.JcrPropertyMap#escapeKeyName) and can easily be called 5000-8000 times per page load. Therefore, it adds 10-15% page rendering overhead for Sling-based websites (see attachment). Would it be possible to add either a global cache or at least a session cache for this method? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
