Re: [jira] Commented: (LUCENE-565) Supporting deleteDocuments in IndexWriter (Code and Performance Results Provided)

A strange case I just thought of. Does the new code handle the case where a merge can drop the resulting segment "down a level" (due to deletions)? Example: M=3, B=10, maxMergeDocs=30 1) segment sizes = 30, 30, 30, 30 2) set maxMergeDocs=100 3) add enough docs to cause a merge 4) the leftmos

[jira] Commented: (LUCENE-565) Supporting deleteDocuments in IndexWriter (Code and Performance Results Provided)

[ http://issues.apache.org/jira/browse/LUCENE-565?page=comments#action_12434041 ] Yonik Seeley commented on LUCENE-565: - Thanks for separating out the new merge policy Ning! I'm reviewing the patch now... Assuming everything looks good (it

[jira] Commented: (LUCENE-664) [PATCH] small fixes to the new scoring.html doc

[ http://issues.apache.org/jira/browse/LUCENE-664?page=comments#action_12434029 ] Grant Ingersoll commented on LUCENE-664: OK, I have split this up into the javadocs and the scoring page. All the Query stuff and Similarity stuff is move

Re: After kill -9 index was corrupt

I do have one module that does custom index operations. This is my bulk updater. It creates new index files for the segments it modifies and a new segments file, then uses the same commit mechanism as merging. I.e., it copes its new segments file into "segments" with the commit lock only after a

Re: After kill -9 index was corrupt

I am not stating that you did not uncover a problem. I am only stating that it is not due to OS level caching. Maybe your sequence of events triggered a reread of the index, while some thread was still writing. The reread sees the 'unused segments' and deletes them, and then the other threa

Re: After kill -9 index was corrupt

robert engels wrote on 09/11/2006 07:34 AM: > A kill -9 should not affect the OS's writing of dirty buffers > (including directory modifications). If this were the case, massive > system corruption would almost always occur every time a kill -9 was > used with any program. > > The only thing a kill

Re: After kill -9 index was corrupt

A kill -9 should not affect the OS's writing of dirty buffers (including directory modifications). If this were the case, massive system corruption would almost always occur every time a kill -9 was used with any program. The only thing a kill -9 affects is user level buffering. The OS al

Re: After kill -9 index was corrupt

On Monday 11 September 2006 15:36, Yonik Seeley wrote: > On 9/10/06, Chuck Williams <[EMAIL PROTECTED]> wrote: > > Could a kill -9 prevent data from reaching disk for files that were > > previously closed? > > No. After a close() the OS should have all the data... the process > may be killed but

Re: After kill -9 index was corrupt

On Monday 11 September 2006 09:50, Chuck Williams wrote: > > Paul Elschot wrote on 09/10/2006 09:15 PM: > > On Monday 11 September 2006 02:24, Chuck Williams wrote: > > > >> Hi All, > >> > >> An application of ours under development had a memory link that caused > >> it to slow interminably. O

Re: After kill -9 index was corrupt

Yonik Seeley wrote: On 9/11/06, Michael McCandless <[EMAIL PROTECTED]> wrote: However, I do think it would be a good idea to [optionally] add a sync() call on committing the segments file to still be robust to OS / machine crashing... it would slow down performance of indexing but hopefully not

Re: After kill -9 index was corrupt

On 9/11/06, Michael McCandless <[EMAIL PROTECTED]> wrote: However, I do think it would be a good idea to [optionally] add a sync() call on committing the segments file to still be robust to OS / machine crashing... it would slow down performance of indexing but hopefully not by too much since the

Re: After kill -9 index was corrupt

Yonik Seeley wrote: On 9/10/06, Chuck Williams <[EMAIL PROTECTED]> wrote: Could a kill -9 prevent data from reaching disk for files that were previously closed? No. After a close() the OS should have all the data... the process may be killed but the OS will eventually flush all the buffers, e

Re: After kill -9 index was corrupt

On 9/10/06, Chuck Williams <[EMAIL PROTECTED]> wrote: Could a kill -9 prevent data from reaching disk for files that were previously closed? No. After a close() the OS should have all the data... the process may be killed but the OS will eventually flush all the buffers, etc. File creation is

[jira] Commented: (LUCENE-532) [PATCH] Indexing on Hadoop distributed file system

[ http://issues.apache.org/jira/browse/LUCENE-532?page=comments#action_12433852 ] Chris commented on LUCENE-532: -- Don't mean to resurrect old issues, but we're having the same problem here indexing to DFS and I've applied the patch and it works for

Re: gdata-server newbie installation/code question

On 9/10/06, Peter Decrem <[EMAIL PROTECTED]> wrote: I was able to create the war gdata-server file with simon's documentation. The documentation now states I should create user accounts. Does the section below (authenticate an existing account) also need to be included (in the section Create/upd

gdata-server newbie installation/code question

I was able to create the war gdata-server file with simon's documentation. The documentation now states I should create user accounts. Does the section below (authenticate an existing account) also need to be included (in the section Create/update/delete an account) ? Is the yourAuthenticationst

Re: After kill -9 index was corrupt

Paul Elschot wrote on 09/10/2006 09:15 PM: > On Monday 11 September 2006 02:24, Chuck Williams wrote: > >> Hi All, >> >> An application of ours under development had a memory link that caused >> it to slow interminably. On linux, the application did not response to >> kill -15 in a reasonable

Re: After kill -9 index was corrupt

On Monday 11 September 2006 02:24, Chuck Williams wrote: > Hi All, > > An application of ours under development had a memory link that caused > it to slow interminably. On linux, the application did not response to > kill -15 in a reasonable time, so kill -9 was used to forcibly terminate > it.