Re: Merging database index with fulltext index

2009-03-02 Thread Marcelo Ochoa
Hi: The point to catch with bad performance during merging a database result is to reduce the number of rows visited by your first query. As an example take a look a these two queries using Lucene Domain Index, the two are equivalents: Option A: select * from (select rownum as ntop_pos,q.* fro

Re: Merging database index with fulltext index

2009-03-01 Thread Glen Newton
I would suggest you try LuSql, which was designed specifically to index relational databases into Lucene. It has an extensive user manual/tutorial which has some complex examples involving multi-joins and sub-queries. I am the author of LuSql. LuSql home page: http://lab.cisti-icist.nrc-cnrc.gc.c

Re: Merging database index with fulltext index

2009-03-01 Thread Erick Erickson
I think the message is don't even try unless you're explored the alternatives and found them inadequate. Best Erick On Sun, Mar 1, 2009 at 2:19 AM, wrote: > > Yes. DBSight helps to flatten database objects into Lucene's > > documents. > > OK, thx for the advice. > > But back to my original ques

RE: Merging database index with fulltext index

2009-02-28 Thread spring
> Yes. DBSight helps to flatten database objects into Lucene's > documents. OK, thx for the advice. But back to my original question. When I have to merge both resultsets, what is the best approach to do this? - To unsubscrib

Re: Merging database index with fulltext index

2009-02-28 Thread Chris Lu
Yes. DBSight helps to flatten database objects into Lucene's documents. It's more like Lucene-On-Rails. Custom crawler is supported via java api to crawl outside database. DBSight query syntax and Lucene query syntax are both supported, in addition to customizable analyzer, similarity, ranking, etc

RE: Merging database index with fulltext index

2009-02-28 Thread spring
> Actually you can use DBSight(disclaimer:I work on it) to > collect the data > and keep them in sync. Hm... it fulltext-indexes a database? It supports document content outside the database (custom crawler)? What query-syntax it supports? --

Re: Merging database index with fulltext index

2009-02-28 Thread Chris Lu
Actually you can use DBSight(disclaimer:I work on it) to collect the data and keep them in sync. The free version has most the features and doesn't have size limit. -- Chris Lu - Instant Scalable Full-Text Search On Any Database/Application site: http://www.dbsight.net dem

RE: Merging database index with fulltext index

2009-02-28 Thread spring
> Contrariwise, look for anything by Marcelo Ochoa on the user list > about embedding Lucene in Oracle (which I confess I haven't looked > into at all, but seems interesting). I know this lucene-oracle text cartridge. But my solution has to work with any of the big databases (MS, IBM, Oracle). -

RE: Merging database index with fulltext index

2009-02-28 Thread spring
> I feel this may not be a good example. It was a very simple example. The real database query is very complex and joins serveral tables. It would be an absolute nightmare to copy all these tables into lucene and keep both in sync.

Re: Merging database index with fulltext index

2009-02-28 Thread Erick Erickson
I'll second Chris's comment and ask whether you've considered denormalizing your data into Lucene and sticking exclusively with Lucene? Contrariwise, look for anything by Marcelo Ochoa on the user list about embedding Lucene in Oracle (which I confess I haven't looked into at all, but seems intere

Re: Merging database index with fulltext index

2009-02-28 Thread Chris Lu
I feel this may not be a good example. Since you can easily index field c, a, d and let Lucene to handle the filter "c = 'foo'" and the order by clause"order by a desc, d" -- Chris Lu - Instant Scalable Full-Text Search On Any Database/Application site: http://www.dbsight.

Merging database index with fulltext index

2009-02-28 Thread spring
Hi, what is the best approach to merge a database index with a lucene fulltext index? Both databases store a unique ID per doc. This is the join criteria. requirements: * both resultsets may be very big (100.000 and much more) * the merged resultset must be sorted by database index and/or releva