Compound / non-compound index files and SIGKILL

2006-06-05 Thread Rob Staveley (Tom)
I've been indexing live data into a compound index from an MTA. I'm resolving a bunch of problems unrelated to Lucene (disparate hangs in my content handlers). When I get a hang, I typically need to kill my daemon, alas more often than not using kill -9 (SIGKILL). However, these SIGKILLs are leavi

Re: Maybe a bug of lucene 1.9

2006-06-05 Thread hu andy
I am very grateful for all your reply and very sorry for my late response. You can see that I posted my message twice, because I didn't see it after I posted first and thought it wouldn't appear in the list. So these days I didn't check my gmail box. I have figured out that problem. The index was

Re: MMapDirectory vs RAMDirectory

2006-06-05 Thread Peter Keegan
I'm reposting this from java-dev to java-user for greater exposure. My search process is using MMapDirectory on a read-only index via: -Dorg.apache.lucene.FSDirectory.class=org.apache.lucene.store.MMapDirectory Another indexing process is building the next version of the index in a different di

Re: query parser field name aliases

2006-06-05 Thread karl wettin
On Mon, 2006-06-05 at 10:55 +1000, Daniel Noll wrote: > Erik Hatcher wrote: > > > > On Jun 4, 2006, at 5:57 PM, karl wettin wrote: > >> I was thinking it could be nice if the query parser handled aliases by > > > > Rather than hacking QueryParser, simply do some regex replaces on the > > string

duplicate results MultiFieldQueryParser

2006-06-05 Thread varun sood
Hi, I am searching two fields at a time and using MultiFieldQueryParser to parse my query (method described in In Action book). I noticed that I am getting duplicate results. The document exists only once in the index structure but in results its showing the same doucment multiple times with same

RE: Compound / non-compound index files and SIGKILL

2006-06-05 Thread Charles.Sanders
Rob, Not sure I have the answer to your question, but thought I would add my observations. According to the book "Lucene In Action", the default index type is compound, although I did not find this mentioned in the API docs. However, my observations have been that this is not true. At least on m

Re: duplicate results MultiFieldQueryParser

2006-06-05 Thread Erick Erickson
A few thoughts... 1> are you sure you only indexed the document once? If you indexed the same data multiple times, you'll have duplicate documents, each of which will have a different Lucene ID (i.e. doc()). 2> have you examined your index with, say, Luke? I've found that a wonderful tool for se

RE: Compound / non-compound index files and SIGKILL

2006-06-05 Thread Rob Staveley (Tom)
Thanks for the suggestion, Charles. I've also been consistently explicitly setting writer.setUseCompoundFile(true) too. When I started the project with Lucene 1.4.3 and ran out of file handles without it I found that it was needed. I'm fairly certain from timestamps on the orphaned files that the o

Re: Compound / non-compound index files and SIGKILL

2006-06-05 Thread Otis Gospodnetic
Rob, My guess is that those are orphans that you cannot save and add to your index. Who knows what state things are in if you killed the JVM. I think you could try editing your segments file and manually adding all segments to it (e.g. _15djq from your example), and then try optimizing the ind

Re: query parser field name aliases

2006-06-05 Thread Erik Hatcher
On Jun 5, 2006, at 11:07 AM, karl wettin wrote: On Mon, 2006-06-05 at 10:55 +1000, Daniel Noll wrote: Erik Hatcher wrote: On Jun 4, 2006, at 5:57 PM, karl wettin wrote: I was thinking it could be nice if the query parser handled aliases by Rather than hacking QueryParser, simply do some r

lucene-web demo is not working?

2006-06-05 Thread Adam Saeed
I have not able to fix the below errors, for lucene-web demo. installed tomcat-5.5, lucene-2.0.0 and jdk 1.4. after enter search button error shows: http status 500 org.apache.jasper.JasperException: Unable to compile class for JSP An error occurred at line: 60 in the jsp file: /results.jsp Gene

lucene with mysql integration

2006-06-05 Thread Adam Saeed
Can some body help me to integrate without coding all the stuff ; lucene with mysql. is there any startup document for a Newbie Java person for lucene with mysql? Thanks Adam - To unsubscribe, e-mail: [EMAIL PROTECTED] For add

Re: query parser field name aliases

2006-06-05 Thread karl wettin
On Mon, 2006-06-05 at 12:58 -0400, Erik Hatcher wrote: > > Huh? Could you elaborate on what won't work about subclassing? I would for instance have to add the same chunk of code to my QueryParser- and my MultiFieldQueryParser-subclass.

Re: lucene-web demo is not working?

2006-06-05 Thread Daniel Naber
On Montag 05 Juni 2006 19:16, Adam Saeed wrote: > Generated servlet error: > The method parse(String) in the type QueryParser is not applicable for > the arguments (String, String, Analyzer) There's a bug in that file. You can download the fixed version here: http://svn.apache.org/viewvc/lucene/

RE: Compound / non-compound index files and SIGKILL

2006-06-05 Thread Rob Staveley (Tom)
Thanks for the advice, Otis. It makes sense that they are not to be trusted after the brutality of SIGKILL (the untrappable signal). Alas, SIGTERM wasn't breaking out of my hang. I'll see what's snagging it in my ShutdownHooks, because I need to be able to handle hangs one way or another, because t

Re: Compound / non-compound index files and SIGKILL

2006-06-05 Thread Volodymyr Bychkoviak
Hi. My five cents :) It might be helpful to know how lucene is working with compound files. When segment is flushed to disk it is written uncompound and after that is merged into single .cfs file. If you don't change default setting for using compound files (which is on) this is only place (I

Re: lucene with mysql integration

2006-06-05 Thread Chris Lu
Basically you select all your data and use Lucene's simple API to create a Document object. But this is just creating index part. To parse the user-input query and maintain content~lucene index synchronization needs some effort. You can try DBSight. It can setup a database search maybe 15 minutes

Re: query parser field name aliases

2006-06-05 Thread Chris Hostetter
: There is a problem with this. Java does not allow multiple inheritance : and there are other subclasses of QueryParser. Perhaps I can fix it with : a decorator instead. you might be able to go with a Delegator/Wrapper .. subclass QueryParser to inherit the grammer, override all of the normal me

Re: query parser field name aliases

2006-06-05 Thread Yonik Seeley
Karl, taking this in context with your other threads, it seems like you are tring to insert extra functionality at many different points in lucene. Would some type of AOP work for you? -Yonik On 6/5/06, karl wettin <[EMAIL PROTECTED]> wrote: On Mon, 2006-06-05 at 12:58 -0400, Erik Hatcher wrot

Re: lucene-web demo is not working?

2006-06-05 Thread Adam Saeed
Thanks, its running fine now. Thanks again Adam Daniel Naber wrote: On Montag 05 Juni 2006 19:16, Adam Saeed wrote: Generated servlet error: The method parse(String) in the type QueryParser is not applicable for the arguments (String, String, Analyzer) There's a bug in that file. You

Re: query parser field name aliases

2006-06-05 Thread karl wettin
On Mon, 2006-06-05 at 10:41 -0700, Chris Hostetter wrote: > > you might be able to go with a Delegator/Wrapper .. subclass > QueryParser to inherit the grammer, override all of the normal methods > you override in a subclass to include your field aliasing, but instead > of defering to super defer

Re: query parser field name aliases

2006-06-05 Thread karl wettin
On Mon, 2006-06-05 at 13:45 -0400, Yonik Seeley wrote: > Karl, taking this in context with your other threads, it seems like > you are tring to insert extra functionality at many different points > in lucene. Would some type of AOP work for you? I'm not sure I know what you mean. Try to decouple

Re: query parser field name aliases

2006-06-05 Thread Chris Hostetter
: > you might be able to go with a Delegator/Wrapper .. subclass : > QueryParser to inherit the grammer, override all of the normal methods : > you override in a subclass to include your field aliasing, but instead : > of defering to super defer to another instance of QueryParser : > specified at

RE: Compound / non-compound index files and SIGKILL

2006-06-05 Thread Rob Staveley (Tom)
This is a real eye-opener, Volodymyr. Many thanks. I guess that means that my orphan-producing hangs must be addDocument() calls, and not in the content handlers, as I'd previously assumed. I'll put some debug before and after my addDocument() calls to confirm (and point my writer's infoStream to S

Re: duplicate results MultiFieldQueryParser

2006-06-05 Thread varun sood
On 6/5/06, Erick Erickson <[EMAIL PROTECTED]> wrote: A few thoughts... 1> are you sure you only indexed the document once? If you indexed the same data multiple times, you'll have duplicate documents, each of which will have a different Lucene ID (i.e. doc()). Yes.. but I will make sure agai

Re: duplicate results MultiFieldQueryParser

2006-06-05 Thread Erick Erickson
Even if the index is huge, you should be able to open it in Luke and submit a search just like the one that's giving you trouble, and then examine the documents. Yes, it'll take a while to open the index, but not an outrageous amount of time... unless you run out of memory or something. Best Eric

Re: query parser field name aliases

2006-06-05 Thread karl wettin
On Mon, 2006-06-05 at 11:45 -0700, Chris Hostetter wrote: > thought "Wrapper" was an official pattern name It is. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]

Re: MMapDirectory vs RAMDirectory

2006-06-05 Thread Daniel Noll
Peter Keegan wrote: There is no 'unmap' method, so my understanding is that the file mapping is valid until the underlying buffer is garbage-collected. However, forcing the gc doesn't help. You're half right. The file mapping is indeed valid until the underlying buffer is garbage collected,

spring & lucene

2006-06-05 Thread Rajiv Roopan
Hello, I'm using the spring framework to define my indexsearcher and indexwriter. They are defined as persistent singletons in the application. I'm quite new to lucene and I realize that after updating an index using IndexWriter you nead to re-create an instance of IndexSearcher to search that

Browse Functionality

2006-06-05 Thread muraalee
Hi , I am looking at exposing a "Browse" like functionality using lucene. Assume this scenario, a user Browses a Field ( Untokenized one like Book Title ) and he sees a list of "terms" [Book titles ] and their associated doc Hits ( Number of articles / documents this term appears in ). I believe t