[jira] [Resolved] (OAK-4383) Benchmarks tests for oak-auth-external

2016-06-03 Thread angela (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

angela resolved OAK-4383.
-
   Resolution: Fixed
Fix Version/s: 1.5.4

> Benchmarks tests for oak-auth-external
> --
>
> Key: OAK-4383
> URL: https://issues.apache.org/jira/browse/OAK-4383
> Project: Jackrabbit Oak
>  Issue Type: Epic
>  Components: auth-external, run
>Reporter: angela
> Fix For: 1.5.4
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (OAK-4418) Benchmark Results for Oak 1.4

2016-06-03 Thread angela (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

angela resolved OAK-4418.
-
   Resolution: Fixed
 Assignee: angela
Fix Version/s: 1.5.4

> Benchmark Results for Oak 1.4
> -
>
> Key: OAK-4418
> URL: https://issues.apache.org/jira/browse/OAK-4418
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: auth-external
>Reporter: angela
>Assignee: angela
>Priority: Minor
> Fix For: 1.5.4
>
> Attachments: 
> ExternalAuthentication_ExternalLogin_1.4_fullsync_20160603_181444.csv, 
> ExternalAuthentication_SyncAllExternalUsers_1.4_fullsync_20160603_111919.csv
>
>
> in order to compare the various improvements in oak-auth-external _and_ other 
> modules against the 1.4 state, i would be good to run the benchmarks (at 
> least an adjusted version) against Oak 1.4.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4418) Benchmark Results for Oak 1.4

2016-06-03 Thread angela (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

angela updated OAK-4418:

Fix Version/s: (was: 1.5.4)
   1.4.4

> Benchmark Results for Oak 1.4
> -
>
> Key: OAK-4418
> URL: https://issues.apache.org/jira/browse/OAK-4418
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: auth-external
>Reporter: angela
>Assignee: angela
>Priority: Minor
> Fix For: 1.4.4
>
> Attachments: 
> ExternalAuthentication_ExternalLogin_1.4_fullsync_20160603_181444.csv, 
> ExternalAuthentication_SyncAllExternalUsers_1.4_fullsync_20160603_111919.csv
>
>
> in order to compare the various improvements in oak-auth-external _and_ other 
> modules against the 1.4 state, i would be good to run the benchmarks (at 
> least an adjusted version) against Oak 1.4.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4418) Benchmark Results for Oak 1.4

2016-06-03 Thread angela (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

angela updated OAK-4418:

Attachment: 
ExternalAuthentication_ExternalLogin_1.4_fullsync_20160603_181444.csv

ExternalAuthentication_SyncAllExternalUsers_1.4_fullsync_20160603_111919.csv

Benchmark results for 

- ExternalLoginTest
- SyncAllExternalUsers

with the same test-options as used for benchmarking the dynamic membership and 
the batch-save for the syncmbeanimpl.

> Benchmark Results for Oak 1.4
> -
>
> Key: OAK-4418
> URL: https://issues.apache.org/jira/browse/OAK-4418
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: auth-external
>Reporter: angela
>Priority: Minor
> Attachments: 
> ExternalAuthentication_ExternalLogin_1.4_fullsync_20160603_181444.csv, 
> ExternalAuthentication_SyncAllExternalUsers_1.4_fullsync_20160603_111919.csv
>
>
> in order to compare the various improvements in oak-auth-external _and_ other 
> modules against the 1.4 state, i would be good to run the benchmarks (at 
> least an adjusted version) against Oak 1.4.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4418) Benchmark Results for Oak 1.4

2016-06-03 Thread angela (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314644#comment-15314644
 ] 

angela edited comment on OAK-4418 at 6/3/16 7:25 PM:
-

Benchmark results for 

- ExternalLoginTest
- SyncAllExternalUsers

with the same test-options as used for benchmarking the dynamic membership and 
the batch-save for the syncmbeanimpl (except for the 'dynamicMembership' option 
that is not available in 1.4


was (Author: anchela):
Benchmark results for 

- ExternalLoginTest
- SyncAllExternalUsers

with the same test-options as used for benchmarking the dynamic membership and 
the batch-save for the syncmbeanimpl.

> Benchmark Results for Oak 1.4
> -
>
> Key: OAK-4418
> URL: https://issues.apache.org/jira/browse/OAK-4418
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: auth-external
>Reporter: angela
>Priority: Minor
> Attachments: 
> ExternalAuthentication_ExternalLogin_1.4_fullsync_20160603_181444.csv, 
> ExternalAuthentication_SyncAllExternalUsers_1.4_fullsync_20160603_111919.csv
>
>
> in order to compare the various improvements in oak-auth-external _and_ other 
> modules against the 1.4 state, i would be good to run the benchmarks (at 
> least an adjusted version) against Oak 1.4.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4422) support cluster for FileBlobStore

2016-06-03 Thread Michael Marth (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314362#comment-15314362
 ] 

Michael Marth commented on OAK-4422:


The term "clustering" primarily applies to the node store. Do you use TarMK? If 
yes, what is the deployment architecture? Different independent TarMK-based 
instances sharing one File-system based Datastore (where the binaries live)?

> support cluster for FileBlobStore
> -
>
> Key: OAK-4422
> URL: https://issues.apache.org/jira/browse/OAK-4422
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: blob
>Affects Versions: 1.4.3
>Reporter: Marco Piovesana
>
> I'm using Oak in a system where the user can store arbitrary large binary 
> files and because of that I thought the best option was to use the 
> FileBlobStore as storage subsystem. 
> Now I need to port this solution on a cluster environment, but i saw that 
> clustering is supported only for Mongo and RDBMS storage systems. Is there 
> any plan to suppor it also for the Blob storage? There's a better option?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (OAK-4426) RepositorySidegrade: oak-segment to oak-segment-tar should drop the name length check

2016-06-03 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-4426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomek Rękawek resolved OAK-4426.

Resolution: Fixed

Fixed for trunk in [r1746709|https://svn.apache.org/r1746709].

> RepositorySidegrade: oak-segment to oak-segment-tar should drop the name 
> length check
> -
>
> Key: OAK-4426
> URL: https://issues.apache.org/jira/browse/OAK-4426
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
>
> As mentioned on OAK-4260, this name length verification is causing some data 
> to be dropped from the upgrade. This limitation only affects mongomk 
> deployments, so it should not apply here.
> {code}
> *WARN*  org.apache.jackrabbit.oak.upgrade.nodestate.NameFilteringNodeState - 
> Node name 'node-name' too long.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4426) RepositorySidegrade: oak-segment to oak-segment-tar should drop the name length check

2016-06-03 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-4426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomek Rękawek updated OAK-4426:
---
Fix Version/s: 1.5.4
   1.6

> RepositorySidegrade: oak-segment to oak-segment-tar should drop the name 
> length check
> -
>
> Key: OAK-4426
> URL: https://issues.apache.org/jira/browse/OAK-4426
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Fix For: 1.6, 1.5.4
>
>
> As mentioned on OAK-4260, this name length verification is causing some data 
> to be dropped from the upgrade. This limitation only affects mongomk 
> deployments, so it should not apply here.
> {code}
> *WARN*  org.apache.jackrabbit.oak.upgrade.nodestate.NameFilteringNodeState - 
> Node name 'node-name' too long.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info

2016-06-03 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313969#comment-15313969
 ] 

Tomek Rękawek commented on OAK-4420:


[~alex.parvulescu], I've prepared a naive fix, which copies the /checkpoints 
from the old repository to the new one using the RepositoryCopier, in a similar 
way as the repository content root is copied. It works, but it'll probably 
multiply the target repository size, as the root node will be copied for each 
existing checkpoint.

If I understand correctly, the /checkpoints/.../root node shouldn't contain a 
full copy of the repository but just a reference to a given revision. I'm not 
sure how we can achieve this in the oak-upgrade, which works on the node store 
level and doesn't really know anything about segment or document implementation 
specifics. Any ideas?

> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint 
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Attachments: OAK-4420-naive.patch
>
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take 
> care of moving the checkpoint data and meta. This will save a very expensive 
> full-reindex.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info

2016-06-03 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomek Rękawek updated OAK-4420:
---
Attachment: OAK-4420-naive.patch

> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint 
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
> Attachments: OAK-4420-naive.patch
>
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take 
> care of moving the checkpoint data and meta. This will save a very expensive 
> full-reindex.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (OAK-4165) Too verbose logging during revision gc

2016-06-03 Thread Amit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amit Jain closed OAK-4165.
--

Bulk close for 1.4.3

> Too verbose logging during revision gc
> --
>
> Key: OAK-4165
> URL: https://issues.apache.org/jira/browse/OAK-4165
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Francesco Mari
>Priority: Blocker
>  Labels: cleanup, gc, logging
> Fix For: 1.4, 1.4.3
>
> Attachments: OAK_4165.patch
>
>
> {{FileStore.cleanup}} logs the segment id of any forward reference found when 
> including those in the reference graph. The logged information can amount to 
> several MBs impacting normal operation. Furthermore the actually reclaimed 
> segments are logged, which also makes the log files explode. Finally the 
> processing of the references and individual tar files might be too wordy. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Closed] (OAK-4411) DocumentNodeStore: Improve test coverage for concurrent updates and queries

2016-06-03 Thread Amit Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amit Jain closed OAK-4411.
--

Bulk close for 1.4.3

> DocumentNodeStore: Improve test coverage for concurrent updates and queries
> ---
>
> Key: OAK-4411
> URL: https://issues.apache.org/jira/browse/OAK-4411
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: documentmk
>Affects Versions: 1.4.2, 1.5.2, 1.0.31, 1.2.15
>Reporter: Julian Reschke
>Assignee: Julian Reschke
> Fix For: 1.4.3, 1.5.3, 1.2.16, 1.0.32
>
>
> In the past, we've had several issues related to overlapping update and query 
> operations -- we should have better test coverage for this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (OAK-4426) RepositorySidegrade: oak-segment to oak-segment-tar should drop the name length check

2016-06-03 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-4426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomek Rękawek reassigned OAK-4426:
--

Assignee: Tomek Rękawek

> RepositorySidegrade: oak-segment to oak-segment-tar should drop the name 
> length check
> -
>
> Key: OAK-4426
> URL: https://issues.apache.org/jira/browse/OAK-4426
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
>
> As mentioned on OAK-4260, this name length verification is causing some data 
> to be dropped from the upgrade. This limitation only affects mongomk 
> deployments, so it should not apply here.
> {code}
> *WARN*  org.apache.jackrabbit.oak.upgrade.nodestate.NameFilteringNodeState - 
> Node name 'node-name' too long.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (OAK-4426) RepositorySidegrade: oak-segment to oak-segment-tar should drop the name length check

2016-06-03 Thread Alex Parvulescu (JIRA)
Alex Parvulescu created OAK-4426:


 Summary: RepositorySidegrade: oak-segment to oak-segment-tar 
should drop the name length check
 Key: OAK-4426
 URL: https://issues.apache.org/jira/browse/OAK-4426
 Project: Jackrabbit Oak
  Issue Type: Bug
  Components: segment-tar, upgrade
Reporter: Alex Parvulescu


As mentioned on OAK-4260, this name length verification is causing some data to 
be dropped from the upgrade. This limitation only affects mongomk deployments, 
so it should not apply here.
{code}
*WARN*  org.apache.jackrabbit.oak.upgrade.nodestate.NameFilteringNodeState - 
Node name 'node-name' too long.
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4260) Define and implement migration from oak-segment to oak-segment-tar

2016-06-03 Thread Alex Parvulescu (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313932#comment-15313932
 ] 

Alex Parvulescu edited comment on OAK-4260 at 6/3/16 10:24 AM:
---

another point. I just saw a warning related to very long names, this also 
probably comes from mongo upgrades but seeing as we're moving from segmentmk -> 
segment-tar, the name length should not be an issue, so I think the 
{{NameFilteringNodeState}} should not drop anything.
For reference the longest one I saw was 174 chrs.

{code}
*WARN*  org.apache.jackrabbit.oak.upgrade.nodestate.NameFilteringNodeState - 
Node name 'node-name' too long.
{code}

[edit] created OAK-4426 for this problem


was (Author: alex.parvulescu):
another point. I just saw a warning related to very long names, this also 
probably comes from mongo upgrades but seeing as we're moving from segmentmk -> 
segment-tar, the name length should not be an issue, so I think the 
{{NameFilteringNodeState}} should not drop anything.
For reference the longest one I saw was 174 chrs.

{code}
*WARN*  org.apache.jackrabbit.oak.upgrade.nodestate.NameFilteringNodeState - 
Node name 'node-name' too long.
{code}

> Define and implement migration from oak-segment to oak-segment-tar
> --
>
> Key: OAK-4260
> URL: https://issues.apache.org/jira/browse/OAK-4260
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: segment-tar, segmentmk, upgrade
>Reporter: Michael Dürig
>Assignee: Tomek Rękawek
>  Labels: migration
> Fix For: 1.6
>
>
> We need to come up with a plan, implementation and documentation for how we 
> deal with migrating from {{oak-segment}} to {{oak-segment-next}}. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4260) Define and implement migration from oak-segment to oak-segment-tar

2016-06-03 Thread Alex Parvulescu (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313932#comment-15313932
 ] 

Alex Parvulescu commented on OAK-4260:
--

another point. I just saw a warning related to very long names, this also 
probably comes from mongo upgrades but seeing as we're moving from segmentmk -> 
segment-tar, the name length should not be an issue, so I think the 
{{NameFilteringNodeState}} should not drop anything.
For reference the longest one I saw was 174 chrs.

{code}
*WARN*  org.apache.jackrabbit.oak.upgrade.nodestate.NameFilteringNodeState - 
Node name 'node-name' too long.
{code}

> Define and implement migration from oak-segment to oak-segment-tar
> --
>
> Key: OAK-4260
> URL: https://issues.apache.org/jira/browse/OAK-4260
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: segment-tar, segmentmk, upgrade
>Reporter: Michael Dürig
>Assignee: Tomek Rękawek
>  Labels: migration
> Fix For: 1.6
>
>
> We need to come up with a plan, implementation and documentation for how we 
> deal with migrating from {{oak-segment}} to {{oak-segment-next}}. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313910#comment-15313910
 ] 

Julian Reschke edited comment on OAK-4425 at 6/3/16 9:57 AM:
-

trunk: http://svn.apache.org/r1746696
1.4: http://svn.apache.org/r1746697
1.2: http://svn.apache.org/r1746701
1.0: http://svn.apache.org/r1746702



was (Author: reschke):
trunk: http://svn.apache.org/r1746696
1.4: http://svn.apache.org/r1746697
1.2: http://svn.apache.org/r1746701

> RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39
> 
>
> Key: OAK-4425
> URL: https://issues.apache.org/jira/browse/OAK-4425
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: rdbmk
>Affects Versions: 1.5.2, 1.0.31, 1.2.15, 1.4.3
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
> Fix For: 1.5.3, 1.2.16, 1.0.32, 1.4.4
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-4425:

Labels:   (was: candidate_oak_1_0)

> RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39
> 
>
> Key: OAK-4425
> URL: https://issues.apache.org/jira/browse/OAK-4425
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: rdbmk
>Affects Versions: 1.5.2, 1.0.31, 1.2.15, 1.4.3
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
> Fix For: 1.5.3, 1.2.16, 1.0.32, 1.4.4
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-4425:

Fix Version/s: 1.0.32

> RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39
> 
>
> Key: OAK-4425
> URL: https://issues.apache.org/jira/browse/OAK-4425
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: rdbmk
>Affects Versions: 1.5.2, 1.0.31, 1.2.15, 1.4.3
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
> Fix For: 1.5.3, 1.2.16, 1.0.32, 1.4.4
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313910#comment-15313910
 ] 

Julian Reschke edited comment on OAK-4425 at 6/3/16 9:51 AM:
-

trunk: http://svn.apache.org/r1746696
1.4: http://svn.apache.org/r1746697
1.2: http://svn.apache.org/r1746701


was (Author: reschke):
trunk: http://svn.apache.org/r1746696
1.4: http://svn.apache.org/r1746697


> RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39
> 
>
> Key: OAK-4425
> URL: https://issues.apache.org/jira/browse/OAK-4425
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: rdbmk
>Affects Versions: 1.5.2, 1.0.31, 1.2.15, 1.4.3
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
>  Labels: candidate_oak_1_0
> Fix For: 1.5.3, 1.2.16, 1.4.4
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-4425:

Fix Version/s: 1.2.16

> RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39
> 
>
> Key: OAK-4425
> URL: https://issues.apache.org/jira/browse/OAK-4425
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: rdbmk
>Affects Versions: 1.5.2, 1.0.31, 1.2.15, 1.4.3
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
>  Labels: candidate_oak_1_0
> Fix For: 1.5.3, 1.2.16, 1.4.4
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-4425:

Labels: candidate_oak_1_0  (was: candidate_oak_1_0 candidate_oak_1_2)

> RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39
> 
>
> Key: OAK-4425
> URL: https://issues.apache.org/jira/browse/OAK-4425
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: rdbmk
>Affects Versions: 1.5.2, 1.0.31, 1.2.15, 1.4.3
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
>  Labels: candidate_oak_1_0
> Fix For: 1.5.3, 1.4.4
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-4425:

Fix Version/s: 1.4.4

> RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39
> 
>
> Key: OAK-4425
> URL: https://issues.apache.org/jira/browse/OAK-4425
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: rdbmk
>Affects Versions: 1.5.2, 1.0.31, 1.2.15, 1.4.3
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.5.3, 1.4.4
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-4425:

Labels: candidate_oak_1_0 candidate_oak_1_2  (was: candidate_oak_1_0 
candidate_oak_1_2 candidate_oak_1_4)

> RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39
> 
>
> Key: OAK-4425
> URL: https://issues.apache.org/jira/browse/OAK-4425
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: rdbmk
>Affects Versions: 1.5.2, 1.0.31, 1.2.15, 1.4.3
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.5.3, 1.4.4
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313910#comment-15313910
 ] 

Julian Reschke edited comment on OAK-4425 at 6/3/16 9:46 AM:
-

trunk: http://svn.apache.org/r1746696
1.4: http://svn.apache.org/r1746697



was (Author: reschke):
trunk: http://svn.apache.org/r1746696

> RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39
> 
>
> Key: OAK-4425
> URL: https://issues.apache.org/jira/browse/OAK-4425
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: rdbmk
>Affects Versions: 1.5.2, 1.0.31, 1.2.15, 1.4.3
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
>  Labels: candidate_oak_1_0, candidate_oak_1_2
> Fix For: 1.5.3, 1.4.4
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke resolved OAK-4425.
-
   Resolution: Fixed
Fix Version/s: 1.5.3

trunk: http://svn.apache.org/r1746696

> RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39
> 
>
> Key: OAK-4425
> URL: https://issues.apache.org/jira/browse/OAK-4425
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: rdbmk
>Affects Versions: 1.5.2, 1.0.31, 1.2.15, 1.4.3
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
>  Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4
> Fix For: 1.5.3
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (OAK-4260) Define and implement migration from oak-segment to oak-segment-tar

2016-06-03 Thread Alex Parvulescu (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313889#comment-15313889
 ] 

Alex Parvulescu edited comment on OAK-4260 at 6/3/16 9:35 AM:
--

another discussion point. what happens with external binaries when the 
filedatastore is not present?
We discussed this as a part of OAK-4279 in the context of offline compaction as 
it should work without having the external datastore attached, but what about 
upgrades?
I'm trying one now and it fails because the filedatastore is not there [0], but 
I think it should gracefully handle this, maybe it already does?
For reference the {{SegmentBlob.getReference}} fails, so there's no easy way of 
identifying if a {{SegmentBlob}} is external or not.
[~tomek.rekawek], [~mduerig] thoughts?

[edit] just realized this is the 'old' segmentmk blob 
{{org.apache.jackrabbit.oak.plugins.segment.SegmentBlob}}, ouch.

[0]
{code}
Exception in thread "main" java.lang.RuntimeException: 
javax.jcr.RepositoryException: Failed to copy content
at com.google.common.io.Closer.rethrow(Closer.java:149)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:58)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.main(OakUpgrade.java:42)
Caused by: javax.jcr.RepositoryException: Failed to copy content
at 
org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copy(RepositorySidegrade.java:254)
at 
org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copy(RepositorySidegrade.java:214)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.sidegrade(OakUpgrade.java:69)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:55)
... 1 more
Caused by: java.lang.IllegalStateException: Attempt to read external blob with 
blobId [6c7f61ed907f481c4417c1a31a406707a1f552f1#6720] without specifying 
BlobStore
at 
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.getReference(SegmentBlob.java:129)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeBlob(SegmentWriter.java:646)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeProperty(SegmentWriter.java:763)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeProperty(SegmentWriter.java:750)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNodeUncached(SegmentWriter.java:957)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(SegmentWriter.java:883)
{code}


was (Author: alex.parvulescu):
another discussion point. what happens with external binaries when the 
filedatastore is not present?
We discussed this as a part of OAK-4279 in the context of offline compaction as 
it should work without having the external datastore attached, but what about 
upgrades?
I'm trying one now and it fails because the filedatastore is not there [0], but 
I think it should gracefully handle this, maybe it already does?
For reference the {{SegmentBlob.getReference}} fails, so there's no easy way of 
identifying if a {{SegmentBlob}} is external or not.
[~tomek.rekawek], [~mduerig] thoughts?

[0]
{code}
Exception in thread "main" java.lang.RuntimeException: 
javax.jcr.RepositoryException: Failed to copy content
at com.google.common.io.Closer.rethrow(Closer.java:149)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:58)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.main(OakUpgrade.java:42)
Caused by: javax.jcr.RepositoryException: Failed to copy content
at 
org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copy(RepositorySidegrade.java:254)
at 
org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copy(RepositorySidegrade.java:214)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.sidegrade(OakUpgrade.java:69)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:55)
... 1 more
Caused by: java.lang.IllegalStateException: Attempt to read external blob with 
blobId [6c7f61ed907f481c4417c1a31a406707a1f552f1#6720] without specifying 
BlobStore
at 
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.getReference(SegmentBlob.java:129)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeBlob(SegmentWriter.java:646)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeProperty(SegmentWriter.java:763)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeProperty(SegmentWriter.java:750)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNodeUncached(SegmentWriter.java:957)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(Seg

[jira] [Commented] (OAK-4260) Define and implement migration from oak-segment to oak-segment-tar

2016-06-03 Thread Alex Parvulescu (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313889#comment-15313889
 ] 

Alex Parvulescu commented on OAK-4260:
--

another discussion point. what happens with external binaries when the 
filedatastore is not present?
We discussed this as a part of OAK-4279 in the context of offline compaction as 
it should work without having the external datastore attached, but what about 
upgrades?
I'm trying one now and it fails because the filedatastore is not there [0], but 
I think it should gracefully handle this, maybe it already does?
For reference the {{SegmentBlob.getReference}} fails, so there's no easy way of 
identifying if a {{SegmentBlob}} is external or not.
[~tomek.rekawek], [~mduerig] thoughts?

[0]
{code}
Exception in thread "main" java.lang.RuntimeException: 
javax.jcr.RepositoryException: Failed to copy content
at com.google.common.io.Closer.rethrow(Closer.java:149)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:58)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.main(OakUpgrade.java:42)
Caused by: javax.jcr.RepositoryException: Failed to copy content
at 
org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copy(RepositorySidegrade.java:254)
at 
org.apache.jackrabbit.oak.upgrade.RepositorySidegrade.copy(RepositorySidegrade.java:214)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.sidegrade(OakUpgrade.java:69)
at 
org.apache.jackrabbit.oak.upgrade.cli.OakUpgrade.migrate(OakUpgrade.java:55)
... 1 more
Caused by: java.lang.IllegalStateException: Attempt to read external blob with 
blobId [6c7f61ed907f481c4417c1a31a406707a1f552f1#6720] without specifying 
BlobStore
at 
org.apache.jackrabbit.oak.plugins.segment.SegmentBlob.getReference(SegmentBlob.java:129)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeBlob(SegmentWriter.java:646)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeProperty(SegmentWriter.java:763)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeProperty(SegmentWriter.java:750)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNodeUncached(SegmentWriter.java:957)
at 
org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(SegmentWriter.java:883)
{code}

> Define and implement migration from oak-segment to oak-segment-tar
> --
>
> Key: OAK-4260
> URL: https://issues.apache.org/jira/browse/OAK-4260
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: segment-tar, segmentmk, upgrade
>Reporter: Michael Dürig
>Assignee: Tomek Rękawek
>  Labels: migration
> Fix For: 1.6
>
>
> We need to come up with a plan, implementation and documentation for how we 
> deal with migrating from {{oak-segment}} to {{oak-segment-next}}. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julian Reschke updated OAK-4425:

Labels: candidate_oak_1_0 candidate_oak_1_2 candidate_oak_1_4  (was: )

> RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39
> 
>
> Key: OAK-4425
> URL: https://issues.apache.org/jira/browse/OAK-4425
> Project: Jackrabbit Oak
>  Issue Type: Technical task
>  Components: rdbmk
>Affects Versions: 1.5.2, 1.0.31, 1.2.15, 1.4.3
>Reporter: Julian Reschke
>Assignee: Julian Reschke
>Priority: Trivial
>  Labels: candidate_oak_1_0, candidate_oak_1_2, candidate_oak_1_4
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (OAK-4425) RDBDocumentStore: upgrade MySQL JDBC driver dependency to 5.1.39

2016-06-03 Thread Julian Reschke (JIRA)
Julian Reschke created OAK-4425:
---

 Summary: RDBDocumentStore: upgrade MySQL JDBC driver dependency to 
5.1.39
 Key: OAK-4425
 URL: https://issues.apache.org/jira/browse/OAK-4425
 Project: Jackrabbit Oak
  Issue Type: Technical task
  Components: rdbmk
Affects Versions: 1.4.3, 1.2.15, 1.0.31, 1.5.2
Reporter: Julian Reschke
Assignee: Julian Reschke
Priority: Trivial






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (OAK-4279) Rework offline compaction

2016-06-03 Thread Alex Parvulescu (JIRA)

 [ 
https://issues.apache.org/jira/browse/OAK-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Parvulescu resolved OAK-4279.
--
   Resolution: Fixed
Fix Version/s: (was: 1.6)
   1.5.3

perfect! marking as fixed. thanks [~mduerig] for keeping a close eye.

> Rework offline compaction
> -
>
> Key: OAK-4279
> URL: https://issues.apache.org/jira/browse/OAK-4279
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Alex Parvulescu
>Priority: Blocker
>  Labels: compaction, gc
> Fix For: 1.5.3
>
> Attachments: OAK-4279-binaries.patch, OAK-4279-checkpoints.patch, 
> OAK-4279-v0.patch, OAK-4279-v1.patch, OAK-4279-v2.patch, OAK-4279-v3.patch, 
> OAK-4279-v4.patch
>
>
> The fix for OAK-3348 broke some of the previous functionality of offline 
> compaction:
> * No more progress logging
> * Compaction is not interruptible any more (in the sense of OAK-3290)
> * Offline compaction could remove the ids of the segment node states to 
> squeeze out some extra space. Those are only needed for later generations 
> generated via online compaction. 
> We should probably implement offline compaction again through a dedicated 
> {{Compactor}} class as it was done in {{oak-segment}} instead of relying on 
> the de-duplication cache (aka online compaction). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (OAK-4420) RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint info

2016-06-03 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomek Rękawek reassigned OAK-4420:
--

Assignee: Tomek Rękawek

> RepositorySidegrade: oak-segment to oak-segment-tar should migrate checkpoint 
> info
> --
>
> Key: OAK-4420
> URL: https://issues.apache.org/jira/browse/OAK-4420
> Project: Jackrabbit Oak
>  Issue Type: Bug
>  Components: segment-tar, upgrade
>Reporter: Alex Parvulescu
>Assignee: Tomek Rękawek
>
> The sidegrade from {{oak-segment}} to {{oak-segment-tar}} should also take 
> care of moving the checkpoint data and meta. This will save a very expensive 
> full-reindex.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4279) Rework offline compaction

2016-06-03 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313861#comment-15313861
 ] 

Michael Dürig commented on OAK-4279:


I like looping those flags through the gc options. Good stuff. 

Re. adding the binary record ids to the cache or not I agree to follow up with 
an improvement options. I think the bulk of things for OC is done here. 

> Rework offline compaction
> -
>
> Key: OAK-4279
> URL: https://issues.apache.org/jira/browse/OAK-4279
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Alex Parvulescu
>Priority: Blocker
>  Labels: compaction, gc
> Fix For: 1.6
>
> Attachments: OAK-4279-binaries.patch, OAK-4279-checkpoints.patch, 
> OAK-4279-v0.patch, OAK-4279-v1.patch, OAK-4279-v2.patch, OAK-4279-v3.patch, 
> OAK-4279-v4.patch
>
>
> The fix for OAK-3348 broke some of the previous functionality of offline 
> compaction:
> * No more progress logging
> * Compaction is not interruptible any more (in the sense of OAK-3290)
> * Offline compaction could remove the ids of the segment node states to 
> squeeze out some extra space. Those are only needed for later generations 
> generated via online compaction. 
> We should probably implement offline compaction again through a dedicated 
> {{Compactor}} class as it was done in {{oak-segment}} instead of relying on 
> the de-duplication cache (aka online compaction). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4279) Rework offline compaction

2016-06-03 Thread Alex Parvulescu (JIRA)

[ 
https://issues.apache.org/jira/browse/OAK-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313845#comment-15313845
 ] 

Alex Parvulescu commented on OAK-4279:
--

introduced flags to enable and cap the size of binary content de-duplication 
with http://svn.apache.org/viewvc?rev=1746686&view=rev.

we only have pending the issue related to adding the binary recordids to the 
cache. I see this as an improvement more than a bug, so I'd like to followup in 
a dedicated issue, so we can come back later and collect more numbers for the 
analysis. [~mduerig] agreed?

> Rework offline compaction
> -
>
> Key: OAK-4279
> URL: https://issues.apache.org/jira/browse/OAK-4279
> Project: Jackrabbit Oak
>  Issue Type: Task
>  Components: segment-tar
>Reporter: Michael Dürig
>Assignee: Alex Parvulescu
>Priority: Blocker
>  Labels: compaction, gc
> Fix For: 1.6
>
> Attachments: OAK-4279-binaries.patch, OAK-4279-checkpoints.patch, 
> OAK-4279-v0.patch, OAK-4279-v1.patch, OAK-4279-v2.patch, OAK-4279-v3.patch, 
> OAK-4279-v4.patch
>
>
> The fix for OAK-3348 broke some of the previous functionality of offline 
> compaction:
> * No more progress logging
> * Compaction is not interruptible any more (in the sense of OAK-3290)
> * Offline compaction could remove the ids of the segment node states to 
> squeeze out some extra space. Those are only needed for later generations 
> generated via online compaction. 
> We should probably implement offline compaction again through a dedicated 
> {{Compactor}} class as it was done in {{oak-segment}} instead of relying on 
> the de-duplication cache (aka online compaction). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (OAK-4412) Lucene-memory property index

2016-06-03 Thread JIRA

[ 
https://issues.apache.org/jira/browse/OAK-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313814#comment-15313814
 ] 

Tomek Rękawek commented on OAK-4412:


Patch attached. {{LuceneHybridTest}} presents how it can be used. The only 
change required for the index definition to enable the hybrid feature is adding 
an extra property: {{hybridIndex}} set to {{true}}.

[~chetanm], [~edivad], [~catholicon] - I'd be grateful for some feedback.

> Lucene-memory property index
> 
>
> Key: OAK-4412
> URL: https://issues.apache.org/jira/browse/OAK-4412
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: lucene
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
> Attachments: OAK-4412.patch
>
>
> When running Oak in a cluster, each write operation is expensive. After 
> performing some stress-tests with a geo-distributed Mongo cluster, we've 
> found out that updating property indexes is a large part of the overall 
> traffic.
> The asynchronous index would be an answer here (as the index update won't be 
> made in the client request thread), but the AEM requires the updates to be 
> visible immediately in order to work properly.
> The idea here is to enhance the existing asynchronous Lucene index with a 
> synchronous, locally-stored counterpart that will persist only the data since 
> the last Lucene background reindexing job.
> The new index can be stored in memory or (if necessary) in MMAPed local 
> files. Once the "main" Lucene index is being updated, the local index will be 
> purged.
> Queries will use an union of results from the {{lucene}} and 
> {{lucene-memory}} indexes.
> The {{lucene-memory}} index, as a local stored entity, will be updated using 
> an observer, so it'll get both local and remote changes.
> The original idea has been suggested by [~chetanm] in the discussion for the 
> OAK-4233.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (OAK-4412) Lucene-memory property index

2016-06-03 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/OAK-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomek Rękawek updated OAK-4412:
---
Attachment: OAK-4412.patch

> Lucene-memory property index
> 
>
> Key: OAK-4412
> URL: https://issues.apache.org/jira/browse/OAK-4412
> Project: Jackrabbit Oak
>  Issue Type: Improvement
>  Components: lucene
>Reporter: Tomek Rękawek
>Assignee: Tomek Rękawek
> Fix For: 1.6
>
> Attachments: OAK-4412.patch
>
>
> When running Oak in a cluster, each write operation is expensive. After 
> performing some stress-tests with a geo-distributed Mongo cluster, we've 
> found out that updating property indexes is a large part of the overall 
> traffic.
> The asynchronous index would be an answer here (as the index update won't be 
> made in the client request thread), but the AEM requires the updates to be 
> visible immediately in order to work properly.
> The idea here is to enhance the existing asynchronous Lucene index with a 
> synchronous, locally-stored counterpart that will persist only the data since 
> the last Lucene background reindexing job.
> The new index can be stored in memory or (if necessary) in MMAPed local 
> files. Once the "main" Lucene index is being updated, the local index will be 
> purged.
> Queries will use an union of results from the {{lucene}} and 
> {{lucene-memory}} indexes.
> The {{lucene-memory}} index, as a local stored entity, will be updated using 
> an observer, so it'll get both local and remote changes.
> The original idea has been suggested by [~chetanm] in the discussion for the 
> OAK-4233.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)