RE: Save to database...

2006-01-05 Thread Steven Pannell
Look in the old archive mails and you will find a few people have tried this out. There is even some code around. I have tried this, and to be honest it does not make much sense. The real problem is performance it just takes too long to keep getting the index from the database for performing the

RE: Save to database...

2006-01-05 Thread Aditya Liviandi
:[EMAIL PROTECTED] Sent: Thursday, January 05, 2006 4:22 PM To: 'java-dev@lucene.apache.org' Subject: RE: Save to database... Look in the old archive mails and you will find a few people have tried this out. There is even some code around. I have tried this, and to be honest it does not make much

RE: Save to database...

2006-01-05 Thread Robert Engels
: Save to database... Look in the old archive mails and you will find a few people have tried this out. There is even some code around. I have tried this, and to be honest it does not make much sense. The real problem is performance it just takes too long to keep getting the index from

RE: Save to database...

2006-01-05 Thread Aditya Liviandi
- From: Robert Engels [mailto:[EMAIL PROTECTED] Sent: Thursday, January 05, 2006 4:36 PM To: java-dev@lucene.apache.org Subject: RE: Save to database... There are impl in the contrib that do not need to retrieve the entire index from the db in order to query (there store blocks of files in the db

RE: Save to database...

2006-01-05 Thread Robert Engels
:43 AM To: java-dev@lucene.apache.org; [EMAIL PROTECTED] Subject: RE: Save to database... What I meant is instead of saving the indexes into files, could I save them as tables in a database? I would think there would be a FieldNames table, a TermDictionary table, a TermFrequency table

RE: Save to database...

2006-01-05 Thread Andi Vajda
On Thu, 5 Jan 2006, Aditya Liviandi wrote: Is there anyone who could help me better understand the file structure of the indexes though? The Berkeley DB implementations in the 'db' contrib area write out blocks of index data as fed by the Lucene Directory classes. It has no understanding of

RE: Save to database...

2006-01-05 Thread Andi Vajda
On Thu, 5 Jan 2006, Robert Engels wrote: 4. transactional updates to the index are possible (although index writes are supposedly transactional in std lucene, I have encountered some index corruption with hard failures - I think it is because the files are not synced when flushed/closed).

RE: Save to database...

2006-01-05 Thread Andi Vajda
On Thu, 5 Jan 2006, Robert Engels wrote: Yes, that is what I did in the custom persistence. There are some not so trivial problems to solve though. Normally you cannot seek with BLOBs, (a lot of JDBC/db impl will read the entire BLOB in all cases) so efficiently reading the postings can be

RE: Save to database...

2006-01-05 Thread Robert Engels
' mode would prevent this (at the cost of performance). -Original Message- From: Andi Vajda [mailto:[EMAIL PROTECTED] Sent: Thursday, January 05, 2006 10:32 AM To: java-dev@lucene.apache.org; [EMAIL PROTECTED] Subject: RE: Save to database... On Thu, 5 Jan 2006, Robert Engels wrote: 4

RE: Save to database...

2006-01-05 Thread Andi Vajda
On Thu, 5 Jan 2006, Robert Engels wrote: Well, I think they are supposed to be - that is the reason the segments file is written last. If the segments file is not updated the only problem becomes orphaned files, but the index SHOULD still be consistent. The Lucene index format is quite