Re: Using lucene as a database... good idea or bad idea?

2008-08-03 Thread Marcus Herou
> >> Thanks Andy and Karsten. >> >> - Original Message - From: "Andy Liu" <[EMAIL PROTECTED]> >> To: >> Sent: Thursday, July 31, 2008 8:16 PM >> Subject: Re: Using lucene as a database... good idea or bad idea? >> >> >>

Re: Using lucene as a database... good idea or bad idea?

2008-08-03 Thread Marcus Herou
AIL PROTECTED]>wrote: > Thanks Andy and Karsten. > > - Original Message - From: "Andy Liu" <[EMAIL PROTECTED]> > To: > Sent: Thursday, July 31, 2008 8:16 PM > Subject: Re: Using lucene as a database... good idea or bad idea? > > > > If essen

Re: Using lucene as a database... good idea or bad idea?

2008-08-03 Thread Marcus Herou
will be better option or one database to > > be > > > used as archive and index. > > > > > > One more idea from this list is to use Jackrabbit / JDBM / My SQL to > > > archive > > > the data. Which will be the best? > >

Re: Using lucene as a database... good idea or bad idea?

2008-08-03 Thread Marcus Herou
Hi Don't use JDBM if you want a BTree it is too slow... Been there. Use Xindice's Filer class: org.apache.xindice.core.filer.Filer and instantiate a org.apache.xindice.core.filer.BTreeFiler which is a whole lot faster. JDBM and Xindice are comparable in reading speed but surely not in writing. C

Re: Using lucene as a database... good idea or bad idea?

2008-07-31 Thread Ganesh - yahoo
Thanks Andy and Karsten. - Original Message - From: "Andy Liu" <[EMAIL PROTECTED]> To: Sent: Thursday, July 31, 2008 8:16 PM Subject: Re: Using lucene as a database... good idea or bad idea? If essentially all you need is key-value storage, Berkeley DB for Java work

Re: Using lucene as a database... good idea or bad idea?

2008-07-31 Thread Andy Liu
SQL to > > archive > > the data. Which will be the best? > > > > I am in desiging phase and i have time to explore and prototype any other > > products. Please do suggest me a good one. > > > >

Re: Using lucene as a database... good idea or bad idea?

2008-07-31 Thread Karsten F.
gards > Ganesh > > > -- View this message in context: http://www.nabble.com/Using-lucene-as-a-database...-good-idea-or-bad-idea--tp18703473p18754258.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]

Re: Using lucene as a database... good idea or bad idea?

2008-07-31 Thread Ganesh - yahoo
t; <[EMAIL PROTECTED]> To: Sent: Thursday, July 31, 2008 3:22 PM Subject: Re: Using lucene as a database... good idea or bad idea? Hmmm, I thought it did. Can't say I've studied the code though, so I'll take your word for it. Never mind on the Jackrabbit suggestion :-)

Re: Using lucene as a database... good idea or bad idea?

2008-07-31 Thread Grant Ingersoll
is message in context: http://www.nabble.com/Using-lucene-as-a-database...-good-idea-or-bad-idea--tp18703473p18750334.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: [EMAIL PROTE

Re: Using lucene as a database... good idea or bad idea?

2008-07-31 Thread Karsten F.
look at Apache Jackrabbit, which uses Lucene > underneath as a content repository. > > -Grant > > -- View this message in context: http://www.nabble.com/Using-lucene-as-a-database...-good-idea-or-bad-idea--tp18703473p18750334.html Sent fro

Re: Using lucene as a database... good idea or bad idea?

2008-07-30 Thread Jason Rutherglen
A possible open source solution using a page based database would be to store the documents in http://jdbm.sourceforge.net/ which offers BTree, Hash, and raw page based access. One would use a primary key type of persistent ID to lookup the document data from JDBM. Would be a good Lucene project

Re: Using lucene as a database... good idea or bad idea?

2008-07-30 Thread Marcelo Ochoa
Hi John: Did you test/know Lucene Domain Index for Oracle database? http://marceloochoa.blogspot.com/2007/09/running-lucene-inside-your-oracle-jvm.html If you are using Oracle 10g/11g is completed integrated in Oracle memory space like Oracle Text but based in Lucene. No network round trip i

Re: Using lucene as a database... good idea or bad idea?

2008-07-30 Thread John Evans
Hi All, Thanks for all of the feedback. Largely as a result of the responses I've received from the mailing list, Lucene has made it's way on to our short list of possible solutions. I'm not sure what the timeframe is for implementing a prototype and testing it, but I will try to report back wit

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread Chris Lu
It surely is possible. AFAIK, LinkedIn use lucene to store some data. But, Lucene index in a sense is similar to database index. Both are data structures for a specialized and limited query execution path. So this depends on your applications' query, and how you create the lucene index. The norma

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread Matthew Hall
Yeah.. we do the same thing here for indexes of up to 57M documents (rows), and that's just one part of our implementation. It takes quite a bit of.. wrangling to use lucene in this manner.. but we've found it to be utterly worthwhile. Matt Ian Lea wrote: John I think it's a great idea,

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread Bill Janssen
I do this with uplib (http://uplib.parc.com/) with fair success. Originally I thought I'd need Lucene plus a relational database to store metadata about the documents for metadata searches. So far, though, I've been able to store the metadata in Lucene and use the same Lucene DB for both metadata

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread ನಾಗೇಶ್ ಸುಬ್ರಹ್ಮಣ್ಯ (Nagesh S)
>>>> understand it, but that shouldn't stop you either. You might also have >>>>> >>>> a >>> >>>> look at Apache Jackrabbit, which uses Lucene underneath as a content >>>>> repository. >>>>> >>>&g

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread Grant Ingersoll
a good idea to use Lucene as storage engine? Regards Ganesh - Original Message - From: "Ian Lea" <[EMAIL PROTECTED]> To: Sent: Tuesday, July 29, 2008 2:18 PM Subject: Re: Using lucene as a database... good idea or bad idea? John I think it's a great idea, a

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread Aravind . Yarram
29/2008 10:02 AM Please respond to java-user@lucene.apache.org To java-user@lucene.apache.org cc Subject Re: Using lucene as a database... good idea or bad idea? Hi Ian, Yes, I see that we are discussing an "option" here. But, as I said before (the three parts to search-based sol

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread ನಾಗೇಶ್ ಸುಬ್ರಹ್ಮಣ್ಯ (Nagesh S)
> >> -Grant > >> > >> > >> On Jul 29, 2008, at 5:34 AM, Ganesh - yahoo wrote: > >> > >> Hello all, > >>> > >>> I am also interested in this. I want to archive the content of the > >>> document using Lu

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread Grant Ingersoll
essage - From: "Ian Lea" <[EMAIL PROTECTED]> To: Sent: Tuesday, July 29, 2008 2:18 PM Subject: Re: Using lucene as a database... good idea or bad idea? John I think it's a great idea, and do exactly this to store 5 million+ documents with info that it takes way too l

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread Ian Lea
; Is it a good idea to use Lucene as storage engine? >>> >>> Regards >>> Ganesh >>> >>> - Original Message - From: "Ian Lea" <[EMAIL PROTECTED]> >>> To: >>> Sent: Tuesday, July 29, 2008 2:18 PM >>> Subjec

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread ನಾಗೇಶ್ ಸುಬ್ರಹ್ಮಣ್ಯ (Nagesh S)
using Lucene. >> >> Is it a good idea to use Lucene as storage engine? >> >> Regards >> Ganesh >> >> - Original Message - From: "Ian Lea" <[EMAIL PROTECTED]> >> To: >> Sent: Tuesday, July 29, 2008 2:18 PM >> Subj

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread Grant Ingersoll
: "Ian Lea" <[EMAIL PROTECTED]> To: Sent: Tuesday, July 29, 2008 2:18 PM Subject: Re: Using lucene as a database... good idea or bad idea? John I think it's a great idea, and do exactly this to store 5 million+ documents with info that it takes way too long to get out of our Ora

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread Ganesh - yahoo
8 PM Subject: Re: Using lucene as a database... good idea or bad idea? John I think it's a great idea, and do exactly this to store 5 million+ documents with info that it takes way too long to get out of our Oracle database (think days). Not as many docs as you are talking about, and less dat

Re: Using lucene as a database... good idea or bad idea?

2008-07-29 Thread Ian Lea
John I think it's a great idea, and do exactly this to store 5 million+ documents with info that it takes way too long to get out of our Oracle database (think days). Not as many docs as you are talking about, and less data for each doc, but I wouldn't have any concerns about scaling. There are

Re: Using lucene as a database... good idea or bad idea?

2008-07-28 Thread Hasan Diwan
ubject: Using lucene as a database... good idea or bad idea? Hi All, I have successfully used Lucene in the "tradtiional" way to provide full-text search for various websites. Now I am tasked with developing a data-store to back a web crawler. The crawler can be configured to retrieve a

Using lucene as a database... good idea or bad idea?

2008-07-28 Thread John Evans
Hi All, I have successfully used Lucene in the "tradtiional" way to provide full-text search for various websites. Now I am tasked with developing a data-store to back a web crawler. The crawler can be configured to retrieve arbitrary fields from arbitrary pages, so the result is that each docum