Thanks, Erick and Robert. The reason to use Lucene is we like the robustness and community support for Lucene. We have other components using advanced Lucene features, too.
For one of the component, all we need is an inverted list with a few customized field types. If Lucene can solve it, we prefer to stay with one software package. We can also avoid to reinvent the wheel of segments, NRT updates, etc. In case we want to expand with a bit more sophisticated features, we can do so within Lucene's framework. I will try your suggestions. Thanks again for your helps. On Sat, Aug 24, 2013 at 9:35 AM, Erick Erickson <[email protected]>wrote: > Have you looked at the whole flexible indexing functionality? Here's > a couple of places to start: > > http://www.opensourceconnections.com/2013/06/05/build-your-own-lucene-codec/ > http://www.slideshare.net/LucidImagination/flexible-indexing-in-lucene-40 > > I'm still not quite sure why you want to do this, but have you looked > at FieldInfo? The c'tor allows you to control a lot of things like whether > to store termvectors etc. See: > > http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/index/FieldInfo.html > > And if that doesn't answer, perhaps you'd explain a bit more _why_ you want > to do this, this could be an XY problem. > > Best > Erick > > > On Sat, Aug 24, 2013 at 12:04 PM, Airway Wong <[email protected]> > wrote: > > > Thanks for the suggestion. > > > > We plan to build inverted list for a production system, so there is high > > demand for reliability and performance. > > > > Lucene is a highly sophisticated IR lib and has a lot of features. > Usually > > it is much easier to trim down features and Lucene already starts to > > support customized inverted list. And that's why I am curious if it can > be > > customized to support simple inverted list. > > > > Even for test purpose, it is useful to limit the functionality and > > incrementally add features. My guess is Lucene probably already have a > way > > to support simple inverted index if needed. > > > > Could someone give more insight into this? > > > > Thanks. > > > > > On 08/24/2013 09:27 AM, Airway Wong wrote: > > > > To customize the inverted list for different format, it seems we have > > to > > > > overload many different classes and functions. We are only interested > > in > > > > simple inverted index without position/posting information. > > > > > > > > Is it possible to customize an inverted list format that only support > > > > simple inverted index (keyword -> list of (doc, an integer) pairs), > > without > > > > overloading all classes and functions? > > > > > > Hello! > > > > > > Reducing or making big changes to index functionality points to the > fact > > > that Lucene is a bad choice for you. > > > I would suggest you to try alternatives, especially > http://terrier.org/ > > > (flexible IR system with main goal to serve in academic purposes). > > > > > > > > > Regards, > > > Ivan Krišto > > >
