[
https://issues.apache.org/jira/browse/LUCY-75?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marvin Humphrey updated LUCY-75:
--------------------------------
Attachment: cf_writer_support.diff
CompoundFileWriter.c
CompoundFileWriter.bp
CompoundFileWriter consolidates files within a directory into a combinded
"cf.dat" file, storing their file metadata within a "cfmeta.json" file.
Nested subdirectories and files ending in ".json" are excluded.
The ".json" exclusion policy is necessary because we want our "segmeta.json"
files to be visible when browsing the index; it causes no problems with file
descriptor limits because JSON files tend to be slurped rather than held open.
Each sub-file within the cf.dat file begins on an offset which is a multiple
of 8, allowing memory-mapped regions to be cast to any type of pointer without
memory alignment problems.
The Lucene version of CompoundFileWriter lives underneath
org.apache.lucene.index and operates on a logical index segment; in contrast,
this version lives under Lucy::Store and operates on any subdirectory,
regardless of what files the subdirectory contains.
> CompoundFileWriter
> ------------------
>
> Key: LUCY-75
> URL: https://issues.apache.org/jira/browse/LUCY-75
> Project: Lucy
> Issue Type: Sub-task
> Components: Core
> Reporter: Marvin Humphrey
> Assignee: Marvin Humphrey
> Attachments: cf_writer_support.diff, CompoundFileWriter.bp,
> CompoundFileWriter.c
>
>
> Consolidate files within a subdirectory to save file descriptors.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
