[
https://issues.apache.org/jira/browse/LUCENE-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shai Erera updated LUCENE-5215:
-------------------------------
Attachment: LUCENE-5215.patch
Patch adds FieldInfos generation:
* SegmentInfoPerCommit manages fieldInfosGen; SegmentInfos read/write it, like
delGen.
** Updated SegmentInfos format jdocs
* ReaderAndLiveDocs writes a new FIS generation if there are DV updates, also
updates existing FIs dvGen.
** We now support updating documents in segments where the field wasn't indexed
(sparse DV).
* New Lucene46Codec and Lucene46FieldInfosFormat for writing the dvGen per
field in the fnm file.
** Updated package.html
** Updated FieldInfosFormat jdocs
** Deprecated Lucene45Codec, moved Lucene42FieldInfosWriter to test-framework,
added Lucene45RWCodec
* Added a static utility method to SegmentReader to readFieldInfos from SIPC,
since a couple of places in the code needed to execute same logic.
* Added segmentSuffix to FieldsReader/Writer.
Most of the changes in the patch are due to the new Lucene46Codec. I couldn't
test FIS.gen without making all the other changes (Lucene45Codec deprecation
etc.) because I didn't feel running tests with e.g. -Dtests.codec=Lucene46 is
enough. So the patch is big, but if you want to review the FIS.gen changes, you
should look at: Lucene46Codec, Lucene46FieldInfosFormat, ReaderAndLiveDocs,
SIPC, SIS.
Core tests pass, so I think it's ready for a review. Also, do I understand
correctly that a 4.5 index for TestBackcompat will be created when we release
4.6 (if that issue makes it to 4.6)?
> Add support for FieldInfos generation
> -------------------------------------
>
> Key: LUCENE-5215
> URL: https://issues.apache.org/jira/browse/LUCENE-5215
> Project: Lucene - Core
> Issue Type: New Feature
> Components: core/index
> Reporter: Shai Erera
> Assignee: Shai Erera
> Attachments: LUCENE-5215.patch
>
>
> In LUCENE-5189 we've identified few reasons to do that:
> # If you want to update docs' values of field 'foo', where 'foo' exists in
> the index, but not in a specific segment (sparse DV), we cannot allow that
> and have to throw a late UOE. If we could rewrite FieldInfos (with
> generation), this would be possible since we'd also write a new generation of
> FIS.
> # When we apply NDV updates, we call DVF.fieldsConsumer. Currently the
> consumer isn't allowed to change FI.attributes because we cannot modify the
> existing FIS. This is implicit however, and we silently ignore any modified
> attributes. FieldInfos.gen will allow that too.
> The idea is to add to SIPC fieldInfosGen, add to each FieldInfo a dvGen and
> add support for FIS generation in FieldInfosFormat, SegReader etc., like we
> now do for DocValues. I'll work on a patch.
> Also on LUCENE-5189, Rob raised a concern about SegmentInfo.attributes that
> have same limitation -- if a Codec modifies them, they are silently being
> ignored, since we don't gen the .si files. I think we can easily solve that
> by recording SI.attributes in SegmentInfos, so they are recorded per-commit.
> But I think it should be handled in a separate issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
