[jira] [Commented] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
[ https://issues.apache.org/jira/browse/LUCENE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004715#comment-14004715 ] ASF subversion and git services commented on LUCENE-5636: - Commit 1596582 from [~shaie] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1596582 ] LUCENE-5618, LUCENE-5636: write each DocValues update in a separate file; stop referencing old fieldInfos files > SegmentCommitInfo continues to list unneeded gen'd files > > > Key: LUCENE-5636 > URL: https://issues.apache.org/jira/browse/LUCENE-5636 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Reporter: Shai Erera >Assignee: Shai Erera > Attachments: LUCENE-5636.patch > > > I thought I handled it in LUCENE-5246, but turns out I didn't handle it > fully. I'll upload a patch which improves the test to expose the bug. I know > where it is, but I'm not sure how to fix it without breaking index > back-compat. Can we do that on experimental features? > The problem is that if you update different fields in different gens, the > FieldInfos files of older gens remain referenced (still!!). I open a new > issue since LUCENE-5246 is already resolved and released, so don't want to > mess up our JIRA... > The severity of the bug is that unneeded files are still referenced in the > index. Everything still works correctly, it's just that .fnm files are still > there. But as I wrote, I'm still not sure how to solve it without requiring > apps that use dv updates to reindex. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
[ https://issues.apache.org/jira/browse/LUCENE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004644#comment-14004644 ] ASF subversion and git services commented on LUCENE-5636: - Commit 1596570 from [~shaie] in branch 'dev/trunk' [ https://svn.apache.org/r1596570 ] LUCENE-5618, LUCENE-5636: write each DocValues update in a separate file; stop referencing old fieldInfos files > SegmentCommitInfo continues to list unneeded gen'd files > > > Key: LUCENE-5636 > URL: https://issues.apache.org/jira/browse/LUCENE-5636 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Reporter: Shai Erera >Assignee: Shai Erera > Attachments: LUCENE-5636.patch > > > I thought I handled it in LUCENE-5246, but turns out I didn't handle it > fully. I'll upload a patch which improves the test to expose the bug. I know > where it is, but I'm not sure how to fix it without breaking index > back-compat. Can we do that on experimental features? > The problem is that if you update different fields in different gens, the > FieldInfos files of older gens remain referenced (still!!). I open a new > issue since LUCENE-5246 is already resolved and released, so don't want to > mess up our JIRA... > The severity of the bug is that unneeded files are still referenced in the > index. Everything still works correctly, it's just that .fnm files are still > there. But as I wrote, I'm still not sure how to solve it without requiring > apps that use dv updates to reindex. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
[ https://issues.apache.org/jira/browse/LUCENE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988005#comment-13988005 ] Shai Erera commented on LUCENE-5636: I chatted with Robert about this. The current situation is that the old .fnm files continue to be referenced even when not needed, however when the segment is merged, they go away (as all gen'd files). Given that there's no way to solve it without breaking back-compat, unless we introduce hacks such as checking for a ".fnm" suffix, we discussed how to solve this "going forward". By "going forward" I mean to not change existing segments, but if they contain future updates, write the new information in a better way. Perhaps old .fnm files will still be referenced by those segments, until they're merged away, but new segments will fix that bug. I think that this might be doable together with LUCENE-5618, by writing per-field gen'd DV file, so I'll try to solve it there and if it works I'll resolve that issue as appropriate. > SegmentCommitInfo continues to list unneeded gen'd files > > > Key: LUCENE-5636 > URL: https://issues.apache.org/jira/browse/LUCENE-5636 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Reporter: Shai Erera >Assignee: Shai Erera > Attachments: LUCENE-5636.patch > > > I thought I handled it in LUCENE-5246, but turns out I didn't handle it > fully. I'll upload a patch which improves the test to expose the bug. I know > where it is, but I'm not sure how to fix it without breaking index > back-compat. Can we do that on experimental features? > The problem is that if you update different fields in different gens, the > FieldInfos files of older gens remain referenced (still!!). I open a new > issue since LUCENE-5246 is already resolved and released, so don't want to > mess up our JIRA... > The severity of the bug is that unneeded files are still referenced in the > index. Everything still works correctly, it's just that .fnm files are still > there. But as I wrote, I'm still not sure how to solve it without requiring > apps that use dv updates to reindex. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
[ https://issues.apache.org/jira/browse/LUCENE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986859#comment-13986859 ] Shai Erera commented on LUCENE-5636: Hmm, actually I think I can fix it w/o breaking index back-compat, by manually removing the old .fnm files in SegCommitInfo (I'll also improve how this is done in general). Assuming very little apps, if at all, write their own FieldInfosFormat, I think that's acceptable? I'll work on a patch later. > SegmentCommitInfo continues to list unneeded gen'd files > > > Key: LUCENE-5636 > URL: https://issues.apache.org/jira/browse/LUCENE-5636 > Project: Lucene - Core > Issue Type: Bug > Components: core/index >Reporter: Shai Erera >Assignee: Shai Erera >Priority: Critical > Attachments: LUCENE-5636.patch > > > I thought I handled it in LUCENE-5246, but turns out I didn't handle it > fully. I'll upload a patch which improves the test to expose the bug. I know > where it is, but I'm not sure how to fix it without breaking index > back-compat. Can we do that on experimental features? > The problem is that if you update different fields in different gens, the > FieldInfos files of older gens remain referenced (still!!). I open a new > issue since LUCENE-5246 is already resolved and released, so don't want to > mess up our JIRA... > The severity of the bug is that unneeded files are still referenced in the > index. Everything still works correctly, it's just that .fnm files are still > there. But as I wrote, I'm still not sure how to solve it without requiring > apps that use dv updates to reindex. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org