[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-02 Thread Thomas Mueller (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994348#comment-15994348 ] Thomas Mueller commented on OAK-5192: - An alternative to changing the merge policy might

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-04 Thread Thomas Mueller (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996362#comment-15996362 ] Thomas Mueller commented on OAK-5192: - As discussed with [~teofili] today: * We could u

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-04 Thread Chetan Mehrotra (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996379#comment-15996379 ] Chetan Mehrotra commented on OAK-5192: -- bq. We could use re-enable compression (which a

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-04 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996393#comment-15996393 ] Tommaso Teofili commented on OAK-5192: -- I am a bit concerned about OAK-2808 that we bre

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-04 Thread Chetan Mehrotra (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996398#comment-15996398 ] Chetan Mehrotra commented on OAK-5192: -- bq. break into the Lucene files management algo

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-04 Thread JIRA
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996419#comment-15996419 ] Michael Dürig commented on OAK-5192: Proactively deleting files only helps in controllin

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-04 Thread Chetan Mehrotra (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996427#comment-15996427 ] Chetan Mehrotra commented on OAK-5192: -- bq. It does nothing for the segment store, whi

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-04 Thread JIRA
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996447#comment-15996447 ] Michael Dürig commented on OAK-5192: In our standard longevity test (sites) I analysed h

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-04 Thread Chetan Mehrotra (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996472#comment-15996472 ] Chetan Mehrotra commented on OAK-5192: -- bq. Here content added consists of the number o

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-04 Thread Thomas Mueller (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996610#comment-15996610 ] Thomas Mueller commented on OAK-5192: - > Also with recent changes in OakDirectory files

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-04 Thread Chetan Mehrotra (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996632#comment-15996632 ] Chetan Mehrotra commented on OAK-5192: -- bq. Maybe this is not working as expected? 48'0

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-09 Thread JIRA
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002715#comment-16002715 ] Michael Dürig commented on OAK-5192: The script for generating above diff: {code} impor

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-11 Thread Chetan Mehrotra (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006120#comment-16006120 ] Chetan Mehrotra commented on OAK-5192: -- [~mduerig] As discussed it would be good to hav

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-11 Thread JIRA
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006148#comment-16006148 ] Michael Dürig commented on OAK-5192: >From an offline discussion it seems the best ways

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-11 Thread Chetan Mehrotra (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006202#comment-16006202 ] Chetan Mehrotra commented on OAK-5192: -- [~mduerig] mentioned that FDS config has minRec

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-11 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006209#comment-16006209 ] Tommaso Teofili commented on OAK-5192: -- [~tmueller] I've created OAK-6211 to track that

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-05-11 Thread JIRA
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006445#comment-16006445 ] Michael Dürig commented on OAK-5192: Attached [^binSize100.txt], [^binSize16384.txt], [^

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-06-30 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070165#comment-16070165 ] Tommaso Teofili commented on OAK-5192: -- I am currently testing along the following line

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-07-03 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072183#comment-16072183 ] Tommaso Teofili commented on OAK-5192: -- added link to OAK-6412, although that's not par

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-07-03 Thread JIRA
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072212#comment-16072212 ] Michael Dürig commented on OAK-5192: [~teofili], could you share patches or links to bra

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-07-03 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072225#comment-16072225 ] Tommaso Teofili commented on OAK-5192: -- the above results come from the unit test I've

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-07-03 Thread Chetan Mehrotra (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072317#comment-16072317 ] Chetan Mehrotra commented on OAK-5192: -- {quote} Attached binSize100.txt, binSize16384.t

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-07-03 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072469#comment-16072469 ] Tommaso Teofili commented on OAK-5192: -- bq. So from number it appears that impact of Lu

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-07-03 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072493#comment-16072493 ] Tommaso Teofili commented on OAK-5192: -- I've enhanced the current small benchmark so th

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-07-03 Thread Chetan Mehrotra (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072619#comment-16072619 ] Chetan Mehrotra commented on OAK-5192: -- bq. how can I set this up in a unit test like

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-07-03 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072707#comment-16072707 ] Tommaso Teofili commented on OAK-5192: -- I've tried with the setup suggested by [~chetan

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-07-03 Thread Chetan Mehrotra (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073080#comment-16073080 ] Chetan Mehrotra commented on OAK-5192: -- bq. Perhaps we should also look at the FileDS s

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-07-07 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077882#comment-16077882 ] Tommaso Teofili commented on OAK-5192: -- I've run again my tests, here're the results: |

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-07-27 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16103259#comment-16103259 ] Tommaso Teofili commented on OAK-5192: -- I've done several tests with different paramete

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-08-03 Thread Thomas Mueller (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112358#comment-16112358 ] Thomas Mueller commented on OAK-5192: - The plan is to make the merge policy configurable

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-08-21 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135247#comment-16135247 ] Tommaso Teofili commented on OAK-5192: -- I've taken some of the feedback received on the

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2017-08-23 Thread Tommaso Teofili (JIRA)
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16138934#comment-16138934 ] Tommaso Teofili commented on OAK-5192: -- This is supposed to be the final test run for t

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2016-11-30 Thread JIRA
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15708339#comment-15708339 ] Michael Dürig commented on OAK-5192: See http://blog.mikemccandless.com/2011/02/visuali

[jira] [Commented] (OAK-5192) Reduce Lucene related growth of repository size

2016-11-30 Thread JIRA
[ https://issues.apache.org/jira/browse/OAK-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15708349#comment-15708349 ] Michael Dürig commented on OAK-5192: As a first step for coming up with improved merge s