This one is probably to Mike since he originally wrote this "demo", but sending to dev@ for posteriority.
I'm looking at something similar to what is shown in TestDemoParallelLeafReader -- creating (or recreating) secondary segments on the fly based on primary segments' data. I spent some time going through the code in TestDemoParallelLeafReader to understand how it works and I get the gist of it, but there are certain things I don't quite grasp. 1) What's the purpose of handling all the refcounts on the secondary index LeafReaders? I get there is a cache of those LeafReaders and it sort of updates itself automatically on zero count, but why bother with refcounting at all -- wouldn't it be simpler to assume that when you acquire the ParallelLeafDirectoryReader wrapper everything (primary and secondary leaf readers) are simply closed when the parent is closed? Is the refcounting an optimization for NRT-heavy reopens (where indeed I see the point where those caches may be handy)? 2) I don't get the purpose of keeping closedSegments lookup. See this snippet: > dir.close(); > > // Must do this after dir is closed, else another thread could "rm -rf" while > we are closing > (which makes MDW.close's checkIndex angry): > closedSegments.add(segIDGen); What's odd to be is that dir is closed in the code above (and so it is closed in ParallelReaderClosed hook invoked on leaf reader's close callback, before the segment is added to closedSegments). What does MDW refer to here? Dawid --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
