[
https://issues.apache.org/jira/browse/LUCENE-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12515630
]
Paul Elschot commented on LUCENE-584:
-------------------------------------
Mark,
The exhausted flag is only in the iterator/Matcher, not in the underlying set
data structure. One can use as many iterators as necessary, for example one per
thread, and then there is never a threadsafety problem. (See
BitSetMatcher.getMatcher() which uses a local class for the resulting Matcher.)
You wrote: I use BooleanFilter a lot for security where many large sets are
cached and combined on the fly - caching all the possible combinations as
single bitsets would lead to too many possible combinations.
That can still be done, but one needs to get to the BitSets for example by
caching them outside the Filters and constructing the resulting BitSetMatcher
for the combined Filter on the fly.
An alternative would be to have a BooleanQuery.add(Matcher, Occur), where the
occurrence can only be required or prohibited. Then there is no need to
construct any resulting filter because the boolean logic will be executed
during the search. This might even be more efficient than combining the full
BitSets ahead of the search.
And with many large BitSets cache memory savings from more compact
implementations can also be helpful.
> Decouple Filter from BitSet
> ---------------------------
>
> Key: LUCENE-584
> URL: https://issues.apache.org/jira/browse/LUCENE-584
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.0.1
> Reporter: Peter Schäfer
> Priority: Minor
> Attachments: bench-diff.txt, bench-diff.txt,
> Matcher-core20070725.patch, Matcher-default20070725.patch,
> Matcher-ground20070725.patch, Some Matchers.zip
>
>
> {code}
> package org.apache.lucene.search;
> public abstract class Filter implements java.io.Serializable
> {
> public abstract AbstractBitSet bits(IndexReader reader) throws IOException;
> }
> public interface AbstractBitSet
> {
> public boolean get(int index);
> }
> {code}
> It would be useful if the method =Filter.bits()= returned an abstract
> interface, instead of =java.util.BitSet=.
> Use case: there is a very large index, and, depending on the user's
> privileges, only a small portion of the index is actually visible.
> Sparsely populated =java.util.BitSet=s are not efficient and waste lots of
> memory. It would be desirable to have an alternative BitSet implementation
> with smaller memory footprint.
> Though it _is_ possibly to derive classes from =java.util.BitSet=, it was
> obviously not designed for that purpose.
> That's why I propose to use an interface instead. The default implementation
> could still delegate to =java.util.BitSet=.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
