Hi again, I realized that it is not possible to serialize the RAM index directly to bytes. This is why I made some tests to copy all files in the ram folder to FS folder by iterating over the contents for the RAM folder. With this approach I can create an FS folder holding the index elements. On the other hand, at the deserialization part where I need to create RAM folder directly from the FS folder (again iterating over the contents of the FS folder), I see a problem.
It seems like there are virtual files in the RAM folder where the file names are matching with the entries of cfmeta.json file. When I try to open an indexer or a searcher over the RAM folder, I see an error as following: "File not found: 'mainindex/seg_2/sort-14.ord'". So the RAM folder does not exactly work as the FS folder since it requires the existence of the virtual files in the RAM folder. Why are there these virtual files? (I suspect that for optimization purposes (e.g. in order not to read the cfmeta.json file over again), virtual files hold the cfmeta.json values). So my question is, is it possible to create the virtual files from cfmeta.json value with an API call? Or do you have any other suggestions. Thanks, Serkan On 2018/03/27 22:35:53, [email protected] <[email protected]> wrote: > Hi, > For security reasons we need to work in memory index. This is why we are > working with the RAMFolder instead of FSFolder. It is tested and it seems to > be working good as long as an OOM issue occurs. > > The question is at some point we need to encrypt the index (RAMFolder) and > save it in disk in encrypted format. Adding in memory index to FS index and > then encrypting FS index then deleting the FS index is not an option due to > security reasons. > > So my question is how can we achieve it. Is it possible to > serialize/deserialize the whole RAM folder to a buffer, so it would be > possible for us to encrypt the buffer (or decrypt to a buffer) only? This > would be an ideal solution for us but I could not find any information about > it in the code. > > If this is not an option, is it possible to traverse the RAMFolder and find > the files in folders, then it would be possible for us to create the same > structure where file contents are encrypted individually? > > Please let me know if you need any clarifications > > Thanks, > Serkan >
