[jira] [Updated] (OAK-3253) Support caching in FileDataStoreService
[ https://issues.apache.org/jira/browse/OAK-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashank Gupta updated OAK-3253: Labels: candidate_oak_1_0 candidate_oak_1_2 docs-impacting features performance (was: ) Support caching in FileDataStoreService --- Key: OAK-3253 URL: https://issues.apache.org/jira/browse/OAK-3253 Project: Jackrabbit Oak Issue Type: Improvement Components: blob Affects Versions: 1.3.3 Reporter: Shashank Gupta Assignee: Shashank Gupta Labels: candidate_oak_1_0, candidate_oak_1_2, docs-impacting, features, performance Fix For: 1.3.4 FDS on SAN/NAS storage is not efficient as it involves network call. In OAK. indexes are stored SAN/NAS and even idle system does lot of read system generated data. Enable caching in FDS so the reads are done locally and async upload to SAN/NAS See [previous discussions|https://issues.apache.org/jira/browse/OAK-3005?focusedCommentId=14700801page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14700801] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3254) Log expensive NodeDocument.getNewestRevision() call
Marcel Reutegger created OAK-3254: - Summary: Log expensive NodeDocument.getNewestRevision() call Key: OAK-3254 URL: https://issues.apache.org/jira/browse/OAK-3254 Project: Jackrabbit Oak Issue Type: Improvement Components: core, mongomk Reporter: Marcel Reutegger Assignee: Marcel Reutegger NodeDocument.getNewestRevision() is usually rather quick because it only needs to check changes on the current document. Sometimes it may happen that previous documents are accessed the call is more expensive. I would like to add a debug log when this happens to identify the documents. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702933#comment-14702933
]
Davide Giannella commented on OAK-3251:
---
Didn't look at the actual code yet so a full investigation and check
will be needed; but here's the plan. Can the mentioned ppl share their
opinions?
[~mduerig], [~alex.parvulescu]
{noformat}
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
{noformat}
- add the Fixture concept to the tests (if not already there)
- move them as IT (ie renaming the classes as *IT)
[~egli], [~reschke], [~mreutegg]
{noformat}
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
{noformat}
- add the Fixture concept to the tests (if not already there)
- move them as IT (ie renaming the classes as *IT)
[~tmueller]
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
{noformat}
- does it make sense to move them as ITs?
- if so will move them and see later on about what we can do to
actually speed-up the execution.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-3254) Log expensive NodeDocument.getNewestRevision() call
[ https://issues.apache.org/jira/browse/OAK-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-3254. --- Resolution: Fixed Fix Version/s: 1.3.5 Done in trunk: http://svn.apache.org/r1696578 Log expensive NodeDocument.getNewestRevision() call --- Key: OAK-3254 URL: https://issues.apache.org/jira/browse/OAK-3254 Project: Jackrabbit Oak Issue Type: Improvement Components: core, mongomk Reporter: Marcel Reutegger Assignee: Marcel Reutegger Fix For: 1.3.5 NodeDocument.getNewestRevision() is usually rather quick because it only needs to check changes on the current document. Sometimes it may happen that previous documents are accessed the call is more expensive. I would like to add a debug log when this happens to identify the documents. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702912#comment-14702912
]
Michael Dürig commented on OAK-3251:
Some of those should probably be ITs instead of UTs. Personally I wouldn't mind
to move all of them to ITs.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-1752) Node name queries should use an index
[
https://issues.apache.org/jira/browse/OAK-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703031#comment-14703031
]
Thomas Mueller commented on OAK-1752:
-
OAK-2634 is done now. Please note this patch will need a few changes because of
this (changed name, currently disabled property in the Lucene index)
Node name queries should use an index
-
Key: OAK-1752
URL: https://issues.apache.org/jira/browse/OAK-1752
Project: Jackrabbit Oak
Issue Type: Improvement
Components: query
Reporter: Alex Parvulescu
Assignee: Chetan Mehrotra
Fix For: 1.3.5
Attachments: OAK-1752.patch
The node name queries don't use any index currently, making them really slow
and triggering a lot of traversal warnings.
Simply adding node names to a property index would be too much content
indexed, but as Lucene already indexes the node names, using this index would
be one viable option.
{code}
/jcr:root//*[fn:name() = 'jcr:content']
/jcr:root//*[jcr:like(fn:name(), 'jcr:con%')]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3239) Port RepositoryUpgrade features to the RepositorySidegrade
[ https://issues.apache.org/jira/browse/OAK-3239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702898#comment-14702898 ] Tomek Rękawek commented on OAK-3239: Patch attached. Port RepositoryUpgrade features to the RepositorySidegrade -- Key: OAK-3239 URL: https://issues.apache.org/jira/browse/OAK-3239 Project: Jackrabbit Oak Issue Type: New Feature Components: upgrade Reporter: Tomek Rękawek Attachments: OAK-3239.patch The RepositoryUpgrade class is used to convert CRX2 repository to an Oak node store. Recently it gained a lot of features (repeated upgrades, skipping orphaned versions, filters, etc.) These features are not available in the RepositorySidegrade (used to copy one Oak node store to another one). This is a container task for providing the new features to the RepositorySidegrade. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702943#comment-14702943
]
Michael Dürig commented on OAK-3251:
I'd be fine with moving the segment tests to ITs.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2875) Namespaces keep references to old node states
[ https://issues.apache.org/jira/browse/OAK-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703004#comment-14703004 ] Michael Dürig commented on OAK-2875: You are right. This didn't occur to me earlier. In this case +1 from me for the patch. Namespaces keep references to old node states - Key: OAK-2875 URL: https://issues.apache.org/jira/browse/OAK-2875 Project: Jackrabbit Oak Issue Type: Sub-task Components: core, jcr Reporter: Alex Parvulescu Assignee: Alex Parvulescu Fix For: 1.3.5 Attachments: OAK-2875-v1.patch, OAK-2875-v2.patch As described on the parent issue OA2849, the session namespaces keep a reference to a Tree instance which will make GC inefficient. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-2634) QueryEngine should expose name query as property restriction
[
https://issues.apache.org/jira/browse/OAK-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Mueller resolved OAK-2634.
-
Resolution: Fixed
http://svn.apache.org/r1696587
(oak-core and oak-lucene, where I had to disable index lookup for this property
currently)
QueryEngine should expose name query as property restriction
Key: OAK-2634
URL: https://issues.apache.org/jira/browse/OAK-2634
Project: Jackrabbit Oak
Issue Type: Sub-task
Components: query
Reporter: Chetan Mehrotra
Assignee: Thomas Mueller
Fix For: 1.3.5
Attachments: OAK-2634-with-test.patch, OAK-2634.patch
Currently {{NodeNameImpl}} and {{NodeLocalNameImpl}} do not add restriction
to a filter hence query index cannot handle such queries.
To allow faster execution name related restriction can be converted to a
property restriction
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (OAK-2735) MongoDiffCacheTest.sizeLimit() uses too much memory
[ https://issues.apache.org/jira/browse/OAK-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger resolved OAK-2735. --- Resolution: Not A Problem Fix Version/s: (was: 1.3.5) MongoDiffCache was removed in OAK-3223. This is not a problem anymore. MongoDiffCacheTest.sizeLimit() uses too much memory --- Key: OAK-2735 URL: https://issues.apache.org/jira/browse/OAK-2735 Project: Jackrabbit Oak Issue Type: Test Components: core Reporter: Marcel Reutegger Assignee: Marcel Reutegger Priority: Minor Labels: CI The diff created by the test uses a lot of memory. Either test test should be changed or the implementation should ignore further changes once a threshold is reached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (OAK-3252) make AbstractQueryTest#test() reflects real file paths
[
https://issues.apache.org/jira/browse/OAK-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella resolved OAK-3252.
---
Resolution: Fixed
Fix Version/s: 1.3.5
in trunk at http://svn.apache.org/r1696563
make AbstractQueryTest#test() reflects real file paths
--
Key: OAK-3252
URL: https://issues.apache.org/jira/browse/OAK-3252
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Fix For: 1.3.5
When a failure happens during the execution of {{AbstractQueryTest#test()}}
an error message like the following is shown.
{noformat}
sql2(org.apache.jackrabbit.oak.plugins.index.property.OrderedIndexQueryTest):
Results in target/sql2.txt don't match expected results in
src/test/resources/sql2.txt; compare the files for details
{noformat}
the file paths highlighted don't reflect the actual file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3253) Support caching in FileDataStoreService
[ https://issues.apache.org/jira/browse/OAK-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashank Gupta updated OAK-3253: Fix Version/s: 1.3.4 Support caching in FileDataStoreService --- Key: OAK-3253 URL: https://issues.apache.org/jira/browse/OAK-3253 Project: Jackrabbit Oak Issue Type: Improvement Components: blob Affects Versions: 1.3.3 Reporter: Shashank Gupta Assignee: Shashank Gupta Fix For: 1.3.4 FDS on SAN/NAS storage is not efficient as it involves network call. In OAK. indexes are stored SAN/NAS and even idle system does lot of read system generated data. Enable caching in FDS so the reads are done locally and async upload to SAN/NAS See [previous discussions|https://issues.apache.org/jira/browse/OAK-3005?focusedCommentId=14700801page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14700801] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2896) Putting many elements into a map results in many small segments.
[
https://issues.apache.org/jira/browse/OAK-2896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-2896:
---
Fix Version/s: (was: 1.3.5)
1.4
Putting many elements into a map results in many small segments.
-
Key: OAK-2896
URL: https://issues.apache.org/jira/browse/OAK-2896
Project: Jackrabbit Oak
Issue Type: Bug
Components: segmentmk
Reporter: Michael Dürig
Assignee: Michael Dürig
Labels: performance
Fix For: 1.4
Attachments: OAK-2896.png, OAK-2896.xlsx
There is an issue with how the HAMT implementation
({{SegmentWriter.writeMap()}} interacts with the 256 segment references limit
when putting many entries into the map: This limit gets regularly reached
once the maps contains about 200k entries. At that points segments get
prematurely flushed resulting in more segments, thus more references and thus
even smaller segments. It is common for segments to be as small as 7k with a
tar file containing up to 35k segments. This is problematic as at this point
handling of the segment graph becomes expensive, both memory and CPU wise. I
have seen persisted segment graphs as big as 35M where the usual size is a
couple of ks.
As the HAMT map is used for storing children of a node this might have an
advert effect on nodes with many child nodes.
The following code can be used to reproduce the issue:
{code}
SegmentWriter writer = new SegmentWriter(segmentStore, getTracker(), V_11);
MapRecord baseMap = null;
for (;;) {
MapString, RecordId map = newHashMap();
for (int k = 0; k 1000; k++) {
RecordId stringId =
writer.writeString(String.valueOf(rnd.nextLong()));
map.put(String.valueOf(rnd.nextLong()), stringId);
}
Stopwatch w = Stopwatch.createStarted();
baseMap = writer.writeMap(baseMap, map);
System.out.println(baseMap.size() + + w.elapsed());
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3253) Support caching in FileDataStoreService
[ https://issues.apache.org/jira/browse/OAK-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashank Gupta updated OAK-3253: Component/s: blob Support caching in FileDataStoreService --- Key: OAK-3253 URL: https://issues.apache.org/jira/browse/OAK-3253 Project: Jackrabbit Oak Issue Type: Improvement Components: blob Affects Versions: 1.3.3 Reporter: Shashank Gupta Assignee: Shashank Gupta Fix For: 1.3.4 FDS on SAN/NAS storage is not efficient as it involves network call. In OAK. indexes are stored SAN/NAS and even idle system does lot of read system generated data. Enable caching in FDS so the reads are done locally and async upload to SAN/NAS See [previous discussions|https://issues.apache.org/jira/browse/OAK-3005?focusedCommentId=14700801page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14700801] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3253) Support caching in FileDataStoreService
[ https://issues.apache.org/jira/browse/OAK-3253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashank Gupta updated OAK-3253: Affects Version/s: 1.3.3 Support caching in FileDataStoreService --- Key: OAK-3253 URL: https://issues.apache.org/jira/browse/OAK-3253 Project: Jackrabbit Oak Issue Type: Improvement Components: blob Affects Versions: 1.3.3 Reporter: Shashank Gupta Assignee: Shashank Gupta Fix For: 1.3.4 FDS on SAN/NAS storage is not efficient as it involves network call. In OAK. indexes are stored SAN/NAS and even idle system does lot of read system generated data. Enable caching in FDS so the reads are done locally and async upload to SAN/NAS See [previous discussions|https://issues.apache.org/jira/browse/OAK-3005?focusedCommentId=14700801page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14700801] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3163) Improve binary comparison during repeated upgrades
[
https://issues.apache.org/jira/browse/OAK-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Dürig updated OAK-3163:
---
Assignee: Manfred Baedke
Improve binary comparison during repeated upgrades
--
Key: OAK-3163
URL: https://issues.apache.org/jira/browse/OAK-3163
Project: Jackrabbit Oak
Issue Type: Improvement
Components: upgrade
Affects Versions: 1.2.3, 1.3.3
Reporter: Julian Sedding
Assignee: Manfred Baedke
Labels: performance
Fix For: 1.3.5
Attachments: OAK-3163.patch
OAK-2619 introduced the possibility to run the same upgrade repeatedly and
achieve incremental upgrades. This reduces the time taken for {{CommitHook}}
processing, because the diff is reduced.
However, for incremental upgrades to be really useful, the content-copy phase
needs to be fast. Currently an optimization that was proposed in OAK-2626 was
lost due to the implementation of a better solution to some part of the
problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3239) Port RepositoryUpgrade features to the RepositorySidegrade
[ https://issues.apache.org/jira/browse/OAK-3239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomek Rękawek updated OAK-3239: --- Attachment: OAK-3239.patch Port RepositoryUpgrade features to the RepositorySidegrade -- Key: OAK-3239 URL: https://issues.apache.org/jira/browse/OAK-3239 Project: Jackrabbit Oak Issue Type: New Feature Components: upgrade Reporter: Tomek Rękawek Attachments: OAK-3239.patch The RepositoryUpgrade class is used to convert CRX2 repository to an Oak node store. Recently it gained a lot of features (repeated upgrades, skipping orphaned versions, filters, etc.) These features are not available in the RepositorySidegrade (used to copy one Oak node store to another one). This is a container task for providing the new features to the RepositorySidegrade. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-3239) Port RepositoryUpgrade features to the RepositorySidegrade
[ https://issues.apache.org/jira/browse/OAK-3239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomek Rękawek updated OAK-3239: --- Flags: Patch Port RepositoryUpgrade features to the RepositorySidegrade -- Key: OAK-3239 URL: https://issues.apache.org/jira/browse/OAK-3239 Project: Jackrabbit Oak Issue Type: New Feature Components: upgrade Reporter: Tomek Rękawek Attachments: OAK-3239.patch The RepositoryUpgrade class is used to convert CRX2 repository to an Oak node store. Recently it gained a lot of features (repeated upgrades, skipping orphaned versions, filters, etc.) These features are not available in the RepositorySidegrade (used to copy one Oak node store to another one). This is a container task for providing the new features to the RepositorySidegrade. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3253) Support caching in FileDataStoreService
Shashank Gupta created OAK-3253: --- Summary: Support caching in FileDataStoreService Key: OAK-3253 URL: https://issues.apache.org/jira/browse/OAK-3253 Project: Jackrabbit Oak Issue Type: Improvement Reporter: Shashank Gupta Assignee: Shashank Gupta FDS on SAN/NAS storage is not efficient as it involves network call. In OAK. indexes are stored SAN/NAS and even idle system does lot of read system generated data. Enable caching in FDS so the reads are done locally and async upload to SAN/NAS See [previous discussions|https://issues.apache.org/jira/browse/OAK-3005?focusedCommentId=14700801page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14700801] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3239) Port RepositoryUpgrade features to the RepositorySidegrade
[ https://issues.apache.org/jira/browse/OAK-3239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702907#comment-14702907 ] Tomek Rękawek commented on OAK-3239: [~baedke], could you take a look and merge the patch? Thanks! Port RepositoryUpgrade features to the RepositorySidegrade -- Key: OAK-3239 URL: https://issues.apache.org/jira/browse/OAK-3239 Project: Jackrabbit Oak Issue Type: New Feature Components: upgrade Reporter: Tomek Rękawek Attachments: OAK-3239.patch The RepositoryUpgrade class is used to convert CRX2 repository to an Oak node store. Recently it gained a lot of features (repeated upgrades, skipping orphaned versions, filters, etc.) These features are not available in the RepositorySidegrade (used to copy one Oak node store to another one). This is a container task for providing the new features to the RepositorySidegrade. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702939#comment-14702939
]
Julian Reschke commented on OAK-3251:
-
I'm not convinced. I'd prefer that a regular unit test run on oak-core actually
covers all DocumentStore implementations.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3252) make AbstractQueryTest#test() reflects real file paths
Davide Giannella created OAK-3252:
-
Summary: make AbstractQueryTest#test() reflects real file paths
Key: OAK-3252
URL: https://issues.apache.org/jira/browse/OAK-3252
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
When a failure happens during the execution of {{AbstractQueryTest#test()}} an
error message like the following is shown.
{noformat}
sql2(org.apache.jackrabbit.oak.plugins.index.property.OrderedIndexQueryTest):
Results in target/sql2.txt don't match expected results in
src/test/resources/sql2.txt; compare the files for details
{noformat}
the file paths highlighted don't reflect the actual file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703094#comment-14703094
]
Davide Giannella commented on OAK-3251:
---
[~reschke] are you referring to the Fixture thing or the IT move? Or both? For
the fixture we'll have the test running for all DOCUMENT fixtures; but you'll
have to provide the parameter on the command line as the default clean install
runs SEGMENT_MK.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703106#comment-14703106
]
Julian Reschke commented on OAK-3251:
-
...the fixture. I can see why higher-level projects (say oak-jcr) do not need
to be unit-tested with each MK implementation by default, but I believe the
answer needs to be different for oak-core. A standard unit test run should
exercise at least basic functionality of all code that's in, and that implies
for instance running the same set of tests for each implementation of the
DocumentStore API.
If specific tests are too slow then yes we should check whether we can speed
them up, or make them ITs (BTW: do we have documentation about how to do that
properly? The approach that I adopted from other tests doesn't scale well at
all :-)
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (OAK-3252) make AbstractQueryTest#test() reflects real file paths
[
https://issues.apache.org/jira/browse/OAK-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella reopened OAK-3252:
---
reverted the commit in https://svn.apache.org/r1696596.
It was making oak-lucene breaking the tests.
make AbstractQueryTest#test() reflects real file paths
--
Key: OAK-3252
URL: https://issues.apache.org/jira/browse/OAK-3252
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Fix For: 1.3.5
When a failure happens during the execution of {{AbstractQueryTest#test()}}
an error message like the following is shown.
{noformat}
sql2(org.apache.jackrabbit.oak.plugins.index.property.OrderedIndexQueryTest):
Results in target/sql2.txt don't match expected results in
src/test/resources/sql2.txt; compare the files for details
{noformat}
the file paths highlighted don't reflect the actual file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3042) Suspend commit on external conflict
[ https://issues.apache.org/jira/browse/OAK-3042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcel Reutegger updated OAK-3042: -- Labels: performance (was: ) Suspend commit on external conflict --- Key: OAK-3042 URL: https://issues.apache.org/jira/browse/OAK-3042 Project: Jackrabbit Oak Issue Type: Improvement Components: core, mongomk Reporter: Marcel Reutegger Assignee: Marcel Reutegger Labels: performance A DocumentNodeStore cluster currently shows a conflict behavior, which is not intuitive. A modification may fail with a conflict even though before and after the conflict, the external change is not visible to the current session. There are two aspects to this issue. 1) a modification may conflict with a change done on another cluster node, which is committed but not yet visible on the current cluster node. 2) even after the InvalidItemStateException caused by the conflict, a refreshed session may still not see the external change. The first aspect is a fundamental design decision and cannot be changed easily. The second part can be addressed by suspending the commit until the external conflict becomes visible on the current cluster node. This would at least avoid the awkward situation where the external change is not visible after the InvalidItemStateException. The system would also become more deterministic. A commit currently goes into a number of retries with exponential back off, but there's no guarantee the external modification becomes visible within those retries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OAK-2472) Add support for atomic counters on cluster solutions
[ https://issues.apache.org/jira/browse/OAK-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davide Giannella updated OAK-2472: -- Fix Version/s: (was: 1.3.5) 1.3.6 Add support for atomic counters on cluster solutions Key: OAK-2472 URL: https://issues.apache.org/jira/browse/OAK-2472 Project: Jackrabbit Oak Issue Type: Improvement Components: core Reporter: Davide Giannella Assignee: Davide Giannella Labels: scalability Fix For: 1.3.6 As of OAK-2220 we added support for atomic counters in a non-clustered situation. This ticket is about covering the clustered ones. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OAK-3255) indexing: add advice on indexing jcr:primaryType
Julian Reschke created OAK-3255: --- Summary: indexing: add advice on indexing jcr:primaryType Key: OAK-3255 URL: https://issues.apache.org/jira/browse/OAK-3255 Project: Jackrabbit Oak Issue Type: Task Components: query Reporter: Julian Reschke It seems that it's not always clear whether creating a property index on jcr:primaryType makes sense. As this index is extremely expensive to maintain, it would be good to have documentation advice on this topic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2149) Locks are not enforced
[
https://issues.apache.org/jira/browse/OAK-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702859#comment-14702859
]
Julian Reschke commented on OAK-2149:
-
There are multiple ways to look at this.
a) We shouldn't give the impression that we support proper JCR locking when we
don't, thus adding checks is not a good idea.
b) Even if we can't always enforce locks, we should try to enforce them when we
can.
I started with a), but now I'm leaning towards b): not doing any checking might
cause people to *rely* on this behavior, which might make it even harder to
implement proper locking later on.
Locks are not enforced
--
Key: OAK-2149
URL: https://issues.apache.org/jira/browse/OAK-2149
Project: Jackrabbit Oak
Issue Type: Sub-task
Components: core, jcr
Affects Versions: 1.0.6
Reporter: Damien Obrist
Attachments: 0001-OAK-2149-Locks-are-not-enforced.patch
Locks are not enforced and do not prevent nodes from being modified by
non-lock-owning sessions.
Consider following code excerpt:
{code:java}
Repository repo = ...
String path = /some/node;
Session session1 = repo.login(new SimpleCredentials(A,
password.toCharArray()));
session1.getNode(path).addMixin(mix:lockable);
session1.save();
LockManager lockManager = session1.getWorkspace().getLockManager();
lockManager.lock(path, true, false, Long.MAX_VALUE, null);
Session session2 = repo.login(new SimpleCredentials(B,
password.toCharArray()));
session2.getNode(path).setProperty(foo, bar);
session2.save();
{code}
User {{A}} puts a lock on {{/some/node}}. User {{B}} is still able to set a
property on that node after it has been locked. Instead, this should throw a
{{LockException}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (OAK-2634) QueryEngine should expose name query as property restriction
[
https://issues.apache.org/jira/browse/OAK-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Mueller reassigned OAK-2634:
---
Assignee: Thomas Mueller
QueryEngine should expose name query as property restriction
Key: OAK-2634
URL: https://issues.apache.org/jira/browse/OAK-2634
Project: Jackrabbit Oak
Issue Type: Sub-task
Components: query
Reporter: Chetan Mehrotra
Assignee: Thomas Mueller
Fix For: 1.3.5
Attachments: OAK-2634-with-test.patch, OAK-2634.patch
Currently {{NodeNameImpl}} and {{NodeLocalNameImpl}} do not add restriction
to a filter hence query index cannot handle such queries.
To allow faster execution name related restriction can be converted to a
property restriction
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-1752) Node name queries should use an index
[
https://issues.apache.org/jira/browse/OAK-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702782#comment-14702782
]
Thomas Mueller commented on OAK-1752:
-
The patch looks good to me, I will work on OAK-2634.
Node name queries should use an index
-
Key: OAK-1752
URL: https://issues.apache.org/jira/browse/OAK-1752
Project: Jackrabbit Oak
Issue Type: Improvement
Components: query
Reporter: Alex Parvulescu
Assignee: Chetan Mehrotra
Fix For: 1.3.5
Attachments: OAK-1752.patch
The node name queries don't use any index currently, making them really slow
and triggering a lot of traversal warnings.
Simply adding node names to a property index would be too much content
indexed, but as Lucene already indexes the node names, using this index would
be one viable option.
{code}
/jcr:root//*[fn:name() = 'jcr:content']
/jcr:root//*[jcr:like(fn:name(), 'jcr:con%')]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2634) QueryEngine should expose name query as property restriction
[
https://issues.apache.org/jira/browse/OAK-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702786#comment-14702786
]
Thomas Mueller commented on OAK-2634:
-
Currently running tests, will commit afterwards.
QueryEngine should expose name query as property restriction
Key: OAK-2634
URL: https://issues.apache.org/jira/browse/OAK-2634
Project: Jackrabbit Oak
Issue Type: Sub-task
Components: query
Reporter: Chetan Mehrotra
Assignee: Thomas Mueller
Fix For: 1.3.5
Attachments: OAK-2634-with-test.patch, OAK-2634.patch
Currently {{NodeNameImpl}} and {{NodeLocalNameImpl}} do not add restriction
to a filter hence query index cannot handle such queries.
To allow faster execution name related restriction can be converted to a
property restriction
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3005) OSGI wrapper service for Jackrabbit CachingFDS
[
https://issues.apache.org/jira/browse/OAK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702801#comment-14702801
]
Shashank Gupta commented on OAK-3005:
-
[~tmueller] Make sense.
Ironing out the finer points.
* PID org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore ( with
cacheSize=0 or not set)
-- FDS implementation will be wired and would be compatible to older release.
* PID org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore ( with
cacheSize set )
-- CachingFDS implementation will be wired here.
-- CachingFDS's understands path as cache path (used in S3DS too). So cachepath
would be mapped to CachingFDS's path
-- {{path}} would be mapped CachingFDS's {{fsbackend}} variable.
OSGI wrapper service for Jackrabbit CachingFDS
--
Key: OAK-3005
URL: https://issues.apache.org/jira/browse/OAK-3005
Project: Jackrabbit Oak
Issue Type: Improvement
Components: blob
Affects Versions: 1.0.15
Reporter: Shashank Gupta
Assignee: Shashank Gupta
Labels: candidate_oak_1_0, candidate_oak_1_2, docs-impacting,
features, performance
Fix For: 1.3.1
Attachments: OAK-2729.patch,
org.apache.jackrabbit.oak.plugins.blob.datastore.CachingFDS.sample.config
OSGI service wrapper for JCR-3869 which provides CachingDataStore
capabilities for SAN NAS storage
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3235) Deadlock when closing a concurrently used FileStore
[
https://issues.apache.org/jira/browse/OAK-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-3235:
Affects Version/s: 1.3.3
Deadlock when closing a concurrently used FileStore
---
Key: OAK-3235
URL: https://issues.apache.org/jira/browse/OAK-3235
Project: Jackrabbit Oak
Issue Type: Bug
Components: segmentmk
Affects Versions: 1.3.3
Reporter: Francesco Mari
Priority: Critical
Fix For: 1.3.4
Attachments: OAK-3235-01.patch
A deadlock was detected while stopping the {{SegmentCompactionIT}} using the
exposed MBean.
{noformat}
Found one Java-level deadlock:
=
pool-1-thread-23:
waiting to lock monitor 0x7fa8cf1f0488 (object 0x0007a0081e48, a
org.apache.jackrabbit.oak.plugins.segment.file.FileStore),
which is held by main
main:
waiting to lock monitor 0x7fa8cc015ff8 (object 0x0007a011f750, a
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter),
which is held by pool-1-thread-23
Java stack information for the threads listed above:
===
pool-1-thread-23:
at
org.apache.jackrabbit.oak.plugins.segment.file.FileStore.writeSegment(FileStore.java:948)
- waiting to lock 0x0007a0081e48 (a
org.apache.jackrabbit.oak.plugins.segment.file.FileStore)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.flush(SegmentWriter.java:228)
- locked 0x0007a011f750 (a
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.prepare(SegmentWriter.java:329)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeListBucket(SegmentWriter.java:447)
- locked 0x0007a011f750 (a
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeList(SegmentWriter.java:698)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1190)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1154)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:100)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:85)
at
[jira] [Updated] (OAK-3235) Deadlock when closing a concurrently used FileStore
[
https://issues.apache.org/jira/browse/OAK-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-3235:
Fix Version/s: 1.3.5
Deadlock when closing a concurrently used FileStore
---
Key: OAK-3235
URL: https://issues.apache.org/jira/browse/OAK-3235
Project: Jackrabbit Oak
Issue Type: Bug
Components: segmentmk
Reporter: Francesco Mari
Priority: Critical
Fix For: 1.3.5
Attachments: OAK-3235-01.patch
A deadlock was detected while stopping the {{SegmentCompactionIT}} using the
exposed MBean.
{noformat}
Found one Java-level deadlock:
=
pool-1-thread-23:
waiting to lock monitor 0x7fa8cf1f0488 (object 0x0007a0081e48, a
org.apache.jackrabbit.oak.plugins.segment.file.FileStore),
which is held by main
main:
waiting to lock monitor 0x7fa8cc015ff8 (object 0x0007a011f750, a
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter),
which is held by pool-1-thread-23
Java stack information for the threads listed above:
===
pool-1-thread-23:
at
org.apache.jackrabbit.oak.plugins.segment.file.FileStore.writeSegment(FileStore.java:948)
- waiting to lock 0x0007a0081e48 (a
org.apache.jackrabbit.oak.plugins.segment.file.FileStore)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.flush(SegmentWriter.java:228)
- locked 0x0007a011f750 (a
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.prepare(SegmentWriter.java:329)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeListBucket(SegmentWriter.java:447)
- locked 0x0007a011f750 (a
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeList(SegmentWriter.java:698)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1190)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1154)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:100)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:85)
at
[jira] [Updated] (OAK-3235) Deadlock when closing a concurrently used FileStore
[
https://issues.apache.org/jira/browse/OAK-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Francesco Mari updated OAK-3235:
Fix Version/s: (was: 1.3.5)
1.3.4
Deadlock when closing a concurrently used FileStore
---
Key: OAK-3235
URL: https://issues.apache.org/jira/browse/OAK-3235
Project: Jackrabbit Oak
Issue Type: Bug
Components: segmentmk
Affects Versions: 1.3.3
Reporter: Francesco Mari
Priority: Critical
Fix For: 1.3.4
Attachments: OAK-3235-01.patch
A deadlock was detected while stopping the {{SegmentCompactionIT}} using the
exposed MBean.
{noformat}
Found one Java-level deadlock:
=
pool-1-thread-23:
waiting to lock monitor 0x7fa8cf1f0488 (object 0x0007a0081e48, a
org.apache.jackrabbit.oak.plugins.segment.file.FileStore),
which is held by main
main:
waiting to lock monitor 0x7fa8cc015ff8 (object 0x0007a011f750, a
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter),
which is held by pool-1-thread-23
Java stack information for the threads listed above:
===
pool-1-thread-23:
at
org.apache.jackrabbit.oak.plugins.segment.file.FileStore.writeSegment(FileStore.java:948)
- waiting to lock 0x0007a0081e48 (a
org.apache.jackrabbit.oak.plugins.segment.file.FileStore)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.flush(SegmentWriter.java:228)
- locked 0x0007a011f750 (a
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.prepare(SegmentWriter.java:329)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeListBucket(SegmentWriter.java:447)
- locked 0x0007a011f750 (a
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeList(SegmentWriter.java:698)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1190)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1135)
at
org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:400)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1126)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1154)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:100)
at
org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:85)
at
[jira] [Comment Edited] (OAK-3005) OSGI wrapper service for Jackrabbit CachingFDS
[
https://issues.apache.org/jira/browse/OAK-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702801#comment-14702801
]
Shashank Gupta edited comment on OAK-3005 at 8/19/15 10:08 AM:
---
[~tmueller] Make sense.
Ironing out the finer points.
* PID org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore ( with
cacheSize=0 or not set)
-- FDS implementation will be wired and would be compatible to older release.
* PID org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore ( with
cacheSize set )
-- CachingFDS implementation will be wired here.
-- CachingFDS's understands path as cache path (used in S3DS too). So cachepath
would be mapped to CachingFDS's path
-- {{path}} would be mapped CachingFDS's {{fsbackend}} field.
was (Author: shgupta):
[~tmueller] Make sense.
Ironing out the finer points.
* PID org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore ( with
cacheSize=0 or not set)
-- FDS implementation will be wired and would be compatible to older release.
* PID org.apache.jackrabbit.oak.plugins.blob.datastore.FileDataStore ( with
cacheSize set )
-- CachingFDS implementation will be wired here.
-- CachingFDS's understands path as cache path (used in S3DS too). So cachepath
would be mapped to CachingFDS's path
-- {{path}} would be mapped CachingFDS's {{fsbackend}} variable.
OSGI wrapper service for Jackrabbit CachingFDS
--
Key: OAK-3005
URL: https://issues.apache.org/jira/browse/OAK-3005
Project: Jackrabbit Oak
Issue Type: Improvement
Components: blob
Affects Versions: 1.0.15
Reporter: Shashank Gupta
Assignee: Shashank Gupta
Labels: candidate_oak_1_0, candidate_oak_1_2, docs-impacting,
features, performance
Fix For: 1.3.1
Attachments: OAK-2729.patch,
org.apache.jackrabbit.oak.plugins.blob.datastore.CachingFDS.sample.config
OSGI service wrapper for JCR-3869 which provides CachingDataStore
capabilities for SAN NAS storage
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (OAK-3251) speeding up the build time
Davide Giannella created OAK-3251:
-
Summary: speeding up the build time
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella updated OAK-3251:
--
Description:
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
was:
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702675#comment-14702675
]
Davide Giannella commented on OAK-3251:
---
When running a simple clean install, only the SEGMENT_MK is passed as
fixture. A first quick win I can thing by looking at the tests
namespaces is to adopt the {{FixtureHelper}} in
- org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
- org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
by having them running only for {{DOCUMENT_*}} fixtures. This will
speed up a clean install already by roughly one minute.
On the other hand for speeding up the builds Fixtures oriented builds
on jenkins we could limit to {{SEGMENT_MK}} fixtures the tests
- org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
- org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
- org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
- org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
having therefore a build for {{DOCUMENT_*}} roughly 2 minutes faster.
Thoughts?
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Giannella updated OAK-3251:
--
Attachment: oak-build-times.log
oak-build-for-unittests-times.log
attaching
- [^oak-build-for-unittests-times.log]: the full build output
- [^oak-build-times.log] the parsed version
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702675#comment-14702675
]
Davide Giannella edited comment on OAK-3251 at 8/19/15 8:04 AM:
When running a simple clean install, only the SEGMENT_MK is passed as
fixture. A first quick win I can think by looking at the tests
namespaces is to adopt the {{FixtureHelper}} in
- org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
- org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
by having them running only for {{DOCUMENT_*}} fixtures. This will
speed up a clean install already by roughly one minute.
On the other hand for speeding up the builds Fixtures oriented builds
on jenkins we could limit to {{SEGMENT_MK}} fixtures the tests
- org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
- org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
- org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
- org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
having therefore a build for {{DOCUMENT_*}} roughly 2 minutes faster.
Thoughts?
was (Author: edivad):
When running a simple clean install, only the SEGMENT_MK is passed as
fixture. A first quick win I can thing by looking at the tests
namespaces is to adopt the {{FixtureHelper}} in
- org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
- org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
by having them running only for {{DOCUMENT_*}} fixtures. This will
speed up a clean install already by roughly one minute.
On the other hand for speeding up the builds Fixtures oriented builds
on jenkins we could limit to {{SEGMENT_MK}} fixtures the tests
- org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
- org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
- org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
- org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
having therefore a build for {{DOCUMENT_*}} roughly 2 minutes faster.
Thoughts?
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
