[
https://issues.apache.org/jira/browse/LUCENE-584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558856#action_12558856
]
Michael Busch commented on LUCENE-584:
--------------------------------------
I think I understand now which problems you had when you wanted to
change BooleanFilter and xml-query-parser to use the new Filter APIs.
BooleanFilter is optimized to utilize BitSets for performing boolean
operations fast. Now if we change BooleanFilter to use the new
DocIdSetIterator, then it can't use the fast BitSet operations (e. g.
union for or, intersect for and) anymore.
Now we can introduce BitSetFilter as you suggested and what I did in
the take4 patch. But here's the problem: Introducing subclasses of
Filter doesn't play nicely with the caching mechanism in Lucene.
For example: if we change BooleanFilter to only work with
BitSetFilters, then it won't work with a CachingWrapperFilter anymore,
because CachingWrapperFilter extends Filter. Then we would have to
introduce new CachingWrapper***Filter, for the different kinds of
Filter subclasses, which is a bad design as Mark pointed out in his
comment:
https://issues.apache.org/jira/browse/LUCENE-584?focusedCommentId=12547901#action_12547901
One solution would be to add a getBitSet() method to DocIdBitSet.
DocIdBitSet is a new class that is basically just a wrapper around a
Java BitSet and provides a DocIdSetIterator to access the BitSet.
Then BooleanFilter could do something like this:
{code:java}
DocIdSet docIdSet = filter.getDocIdSet();
if (docIdSet instanceof DocIdBitSet) {
BitSet bits = ((DocIdBitSet) docIdSet).getBitSet();
... // existing code
} else {
throw new UnsupportedOperationException("BooleanFilter only
supports Filters that use DocIdBitSet.");
}
{code}
But then, changing the core filters to use OpenBitSets instead of
Java BitSets is technically an API change, because BooleanFilter
would not work anymore with the core filters.
So if we took this approach we would have to wait until 3.0 to move
the core from BitSet to OpenBitSet and also change BooleanFilter
then to use OpenBitSets. BooleanFilter could then also work with
either of the two BitSet implementions, but probably not with those
two mixed.
Any feedback about this is very welcome. I'll try to further think
about how to marry the new Filter API, caching mechanism and Filter
implementations like BooleanFilter nicely.
> Decouple Filter from BitSet
> ---------------------------
>
> Key: LUCENE-584
> URL: https://issues.apache.org/jira/browse/LUCENE-584
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.0.1
> Reporter: Peter Schäfer
> Assignee: Michael Busch
> Priority: Minor
> Fix For: 2.4
>
> Attachments: bench-diff.txt, bench-diff.txt,
> ContribQueries20080111.patch, lucene-584-take2.patch,
> lucene-584-take3-part1.patch, lucene-584-take3-part2.patch,
> lucene-584-take4-part1.patch, lucene-584-take4-part2.patch, lucene-584.patch,
> Matcher-20070905-2default.patch, Matcher-20070905-3core.patch,
> Matcher-20071122-1ground.patch, Some Matchers.zip, Test20080111.patch
>
>
> {code}
> package org.apache.lucene.search;
> public abstract class Filter implements java.io.Serializable
> {
> public abstract AbstractBitSet bits(IndexReader reader) throws IOException;
> }
> public interface AbstractBitSet
> {
> public boolean get(int index);
> }
> {code}
> It would be useful if the method =Filter.bits()= returned an abstract
> interface, instead of =java.util.BitSet=.
> Use case: there is a very large index, and, depending on the user's
> privileges, only a small portion of the index is actually visible.
> Sparsely populated =java.util.BitSet=s are not efficient and waste lots of
> memory. It would be desirable to have an alternative BitSet implementation
> with smaller memory footprint.
> Though it _is_ possibly to derive classes from =java.util.BitSet=, it was
> obviously not designed for that purpose.
> That's why I propose to use an interface instead. The default implementation
> could still delegate to =java.util.BitSet=.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
