Right, I think Lucene could decide under-the-hood what's the best data
structure when writing the column-stride field. Sort of like how
BitVector has two ways (sparse vs unsparse) of storing itself on disk.
Mike
Otis Gospodnetic wrote:
So that would require Lucene to dynamically/periodically check field
values and their frequencies and switch from a regular inverted
index to a bitmap index or just create an additional bitmap index
for those fields and their values?
Otis
----- Original Message ----
From: Michael McCandless <luc...@mikemccandless.com>
To: java-dev@lucene.apache.org
Sent: Friday, February 27, 2009 4:41:32 PM
Subject: Re: Bitmap index
I think with column stride fields we should use Bitmap Index to
represent fields
that have few values across many docs.
Mike
Uwe Schindler wrote:
In my opinion, we currently use some type of bitmap index with our
filters.
OpenBitSet and SortedVIntList used in filters can be seen as
bitmap indexes
specifying if a document is a hit of the filter or not. Maybe we
can use the
compression technology mentioned in this Wikipedia article to
further
optimize filters and their DocIdSetIterators.
In my opinion, the real use of bitmap indexes is data warehousing,
when
low-cardinality-columns are used. We are using Sybase IQ (a column
oriented
database) that has heavy usage of bitmap indexes (a variation of
them are
called LF - low-fast indexes there).
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de
-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com]
Sent: Friday, February 27, 2009 9:33 PM
To: java-dev@lucene.apache.org
Subject: Bitmap index
Hi,
I've had http://en.wikipedia.org/wiki/Bitmap_index open in my
browser for
weeks, thinking I'd bring it up here -- would a bitmap index make
sense
anywhere in Lucene (or perhaps Solr)?
Otis
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
