RE: Boost value corrupted "read eof exception"

2008-02-15 Thread Allahbaksh Mohammedali Asadullah
Hi Chris and Mike, Thanks for your help. Chis you got me right for both points. Sorry that I could not put my queries properly. As I can't set and get boost value I have to do my implementation some other way. Please help me in the same I have a three column table which I am trying to save i

Re: How to pass additional information into Similarity.scorePayload(...)

2008-02-15 Thread Cedric Ho
Thanks ~ Yes it seems this would be quite difficult to achieve with Lucene. Nevermind, I'll try to figure out a workaround for it. Thanks for helping =) Cedric On Feb 16, 2008 5:30 AM, Paul Elschot <[EMAIL PROTECTED]> wrote: > Hi Cedric, > > I think I'm beginning to get the point of the [10/5/2

Re: DefaultIndexAccessor

2008-02-15 Thread vivek sar
Mark, Thanks for the quick fix. Actually, it is possible that there might had been simultaneous queries using the MultiSearcher. I assumed it was thread-safe, thus was re-using the same instance. I'll update my application code as well. Thanks, -vivek On Feb 15, 2008 5:56 PM, Mark Miller <[EMAIL

Re: DefaultIndexAccessor

2008-02-15 Thread vivek sar
Mark, Here is the scenario when I saw this exception, 1) A search was run which uses MultiSearcher. This search took more than 3 mins to complete (due to index size and multiple indices) 2) Just a minute after the search was started, we started writing (in a separate thread) to one of the index

Re: DefaultIndexAccessor

2008-02-15 Thread Mark Miller
Here is the fix: https://issues.apache.org/jira/browse/LUCENE-1026 vivek sar wrote: Mark, There seems to be some issue with DefaultMultiIndexAccessor.java. I got following NPE exception, 2008-02-13 07:10:28,021 ERROR [http-7501-Processor6] ReportServiceImpl - java.lang.NullPointerExcep

Re: DefaultIndexAccessor

2008-02-15 Thread Mark Miller
Okay, sorry about this one vivek. Added to the unit tests to expose this. When I took out the MultiSearcher caching, I kept the concept of sharing a single MultiIndexAccessor. Unfortunately, this meant that multiple threads were sharing the same Searcher to Accessor Map that was used to track w

Re: DefaultIndexAccessor

2008-02-15 Thread Mark Miller
Hey vivek, sorry to hear you are having problems. I am trying to figure out how you may be seeing this problem. The IndexAccessor cannot return null because you would get an IllegalStateException not a NullPointerException. Also, the released MultiSearcher cannot be null because the Exception

Re: DefaultIndexAccessor

2008-02-15 Thread vivek sar
Mark, There seems to be some issue with DefaultMultiIndexAccessor.java. I got following NPE exception, 2008-02-13 07:10:28,021 ERROR [http-7501-Processor6] ReportServiceImpl - java.lang.NullPointerException at org.apache.lucene.indexaccessor.DefaultMultiIndexAccessor.release(Defa

Re: Searching for multiple criteria (accross 2 tables)

2008-02-15 Thread Jake Mannix
What the other posters are referring to is that you will have to probably write some java code to do lucene indexing: you can get access to your model objects (with all their dependent data) in java. - since you are using hibernate, this shouild be easy- then create lucene documents from your mode

Re: How to pass additional information into Similarity.scorePayload(...)

2008-02-15 Thread Paul Elschot
Hi Cedric, I think I'm beginning to get the point of the [10/5/2], and why you called that requirement a bit strange, see below. To use both normal position info and paragraph position info you'll need two separate, one normal, and one for the paragraphs. To use the normal field to determine the

Re: Searching for multiple criteria (accross 2 tables)

2008-02-15 Thread Chris Lu
Sorry, sent the previous draft email by mistake. Here is the correct one. Sounds a typical SQL pivot problem. select Id, SIN, data.* from IdCard, (SELECT ID MAX(CASE WHEN name = 'Fname' THEN Value END) AS Fname, MAX(CASE WHEN name = 'Lname' THEN Value END) AS Lname, MAX(CA

Re: Boost value corrupted "read eof exception"

2008-02-15 Thread Chris Hostetter
Reading between the lines, I suspect two things are happening here, both of which seems like missunderstandings ... 1) i think the IOException seen while debugging is refering to an exception which is handled internally in Lucene and is not actually a problem -- but Allahbaksh is seeing it whe

Re: Boost value corrupted "read eof exception"

2008-02-15 Thread Michael McCandless
Allahbaksh Mohammedali Asadullah <[EMAIL PROTECTED]> wrote: > > Hi, > Thanks for your valuable time.Could you get the same boost value what you > had stored. In my case I always get boost value as 1 ie. Default value. When > I debug it goes through IOException though it does not show any Exception

Re: Searching for multiple criteria (accross 2 tables)

2008-02-15 Thread lmctndi
Thanks for your reply. Your idea prompts more questions: I understand what you are saying but don't know how to implement it. How do you go about joining all rows of all the tables belonging to one person and to index them so that I can actually use "+Fname:john +County:USA" as a query? Eric

Re: Searching for multiple criteria (accross 2 tables)

2008-02-15 Thread Erick Erickson
To expand a bit on Chris's first point: Take off your DB hat and put on your search hat . It sounds like you have simply moved your database tables into Lucene and want to search across them. My rule is that whenever you find yourself trying to make Lucene act like a DB, you need to pause and refle

Re: Searching for multiple criteria (accross 2 tables)

2008-02-15 Thread Chris Lu
Hi, lmctndi, You can liberate yourself from this complicated schema, and use some SQLs to select out the content and put the data into one Lucene index. You may need to avoid Hibernate Search here for your case, which limit you to one way of organizing Lucene indexes. I do think this is where DBS

RE: Boost value corrupted "read eof exception"

2008-02-15 Thread Allahbaksh Mohammedali Asadullah
Hi, Thanks for your valuable time.Could you get the same boost value what you had stored. In my case I always get boost value as 1 ie. Default value. When I debug it goes through IOException though it does not show any Exception on the console in eclipse. Regards, Allahbaksh -Original Mes

Re: Boost value corrupted "read eof exception"

2008-02-15 Thread Michael McCandless
I had to comment out the "search" lines. But then when I run this I don't hit any exception. Tested on Linux & Windows XP. Mike Allahbaksh Mohammedali Asadullah wrote: Hi, I am using lucene 2.3. The exception is properly handled when I run the program by lucene but when I debug it it

Boost value corrupted "read eof exception"

2008-02-15 Thread Allahbaksh Mohammedali Asadullah
Hi, I am using lucene 2.3. The exception is properly handled when I run the program by lucene but when I debug it it hit the exception at addDocuement. Regards, Allahbaksh import java.io.IOException; import java.io.StringReader; import org.apache.lucene.search.Hits; import org.apache.luce

Re: Boost value corrupted "read eof exception"

2008-02-15 Thread Michael McCandless
Which version of Lucene are you using? Can you post the full stack trace for the exception you're hitting? Mike Allahbaksh Mohammedali Asadullah wrote: Hi, I am getting read past eof exception when I am adding Document. The things works fine but while I read the Boost value I get it as

Searching for multiple criteria (accross 2 tables)

2008-02-15 Thread lmctndi
Hi all, [ Sorry for the cryptic title but I can't think of a better one] I am trying to integrate Lucene as a search engine for my database and would appreciate some help on a problem. First, some info about the development platform, I am developing on Windows using Hibernate for DB transaction

Boost value corrupted "read eof exception"

2008-02-15 Thread Allahbaksh Mohammedali Asadullah
Hi, I am getting read past eof exception when I am adding Document. The things works fine but while I read the Boost value I get it as default value i.e 1.0. I tried using RAMDirectory then also I am getting the same error. Can someone throw some light on this. Warm Regards, Allahbaksh **

RE: Design questions

2008-02-15 Thread spring
> You need to watch both the positionincrementgap > (which, as I remember, gets added for each new field of the > same name you add to the document). Make it 0 rather than > whatever it is currently. You may have to create a new analyzer > by subclassing your favorite analyzer and overriding the >

Re: Design questions

2008-02-15 Thread Erick Erickson
You need to watch both the positionincrementgap (which, as I remember, gets added for each new field of the same name you add to the document). Make it 0 rather than whatever it is currently. You may have to create a new analyzer by subclassing your favorite analyzer and overriding the getPositionI

Re: from 2.2.0 to 2.3.0

2008-02-15 Thread Michael McCandless
Sergey Kabashnyuk wrote: On Fri, 15 Feb 2008 16:31:07 +0200, Michael McCandless <[EMAIL PROTECTED]> wrote: OK I found one case that can cause the assert to trip incorrectly. Are you using empty string field names? Not. May be empty content of the field. Hmmm, ok. Empty content is fine

Which parser or query should I be using

2008-02-15 Thread Mitchell, Erica
trying to send this again... i keep getting delivery failures so i've attached my code snippet instead My index from Luke looks like this where the columns are for Rank/Field/Text 2 guid 2 1 guid pi1 1 statement Some policy statement for pi3, needs some text with a 2 in it 1 guid pi3

Re: from 2.2.0 to 2.3.0

2008-02-15 Thread Sergey Kabashnyuk
On Fri, 15 Feb 2008 16:31:07 +0200, Michael McCandless <[EMAIL PROTECTED]> wrote: OK I found one case that can cause the assert to trip incorrectly. Are you using empty string field names? Not. May be empty content of the field. I have another question. I see you use asserts in code 2.3.0

Re: from 2.2.0 to 2.3.0

2008-02-15 Thread Michael McCandless
OK I found one case that can cause the assert to trip incorrectly. Are you using empty string field names? I'll open a Jira issue & fix it on the 2.3 branch (so this is fixed in 2.3.1). Thanks for reporting this! Mike Sergey Kabashnyuk wrote: stack trace java.lang.AssertionError

Re: from 2.2.0 to 2.3.0

2008-02-15 Thread Michael McCandless
CheckIndex tool is in the 2.3 JAR. Just make sure the JAR is on your CLASSPATH then run: java org.apache.lucene.index.CheckIndex However, from that stack trace, this is not a corruption issue in your existing index. My guess is you would hit this same exception if you started from a

RE: Design questions

2008-02-15 Thread spring
Well, it seems that this may be a solution for me too. But I'm afraid that someone one day will change this string. And then my app will not work anymore... > -Original Message- > From: Adrian Smith [mailto:[EMAIL PROTECTED] > Sent: Freitag, 15. Februar 2008 13:02 > To: java-user@lucene

Re: from 2.2.0 to 2.3.0

2008-02-15 Thread Sergey Kabashnyuk
stack trace java.lang.AssertionError at org.apache.lucene.index.TermInfosWriter.add(TermInfosWriter.java:143) at org.apache.lucene.index.DocumentsWriter.appendPostings(DocumentsWriter.java:2290) at org.apache.lucene.index.DocumentsWriter.writeSegment(DocumentsWriter.j

Re: from 2.2.0 to 2.3.0

2008-02-15 Thread Michael McCandless
Can you provide the full stack trace, and any details of what you were doing when you tripped that assert? Another thing to try is to run the org.apache.lucene.index.CheckIndex tool (new in 2.3) to check your 2.2 index. It's possible your 2.2 index has some undetected corruption which th

from 2.2.0 to 2.3.0

2008-02-15 Thread Sergey Kabashnyuk
Hi I want to update from lucene 2.2.0 to 2.3.0 but I have assert it TermInfosWriter line 143. What is it mean assert compareToLastTerm(fieldNumber, termText, termTextStart, termTextLength) < 0 || (isIndex && termTextLength == 0 && lastTermTextLength == 0); Can you help me? Sergey Kabashnyuk -

RE: Design questions

2008-02-15 Thread spring
> > Document doc = new Document() > > for (int i = 0; i < pages.length; i++) { > > doc.add(new Field("text", pages[i], Field.Store.NO, > > Field.Index.TOKENIZED)); > > doc.add(new Field("text", "$$", Field.Store.NO, > > Field.Index.UN_TOKENIZED)); > > } > > UN_TOKENIZED. Nice idea!

Re: How to pass additional information into Similarity.scorePayload(...)

2008-02-15 Thread Grant Ingersoll
On Feb 14, 2008, at 3:44 AM, Cedric Ho wrote: Anyway, I know my requirement is a bit strange, so it's ok if I can't do this in Lucene. I'll settle with using a ThreadLocal to store the [10/5/2] weighting and retrieve it in the Similarity.scorePayload(...) function. I don't think you are stil

RE: Design questions

2008-02-15 Thread spring
> Document doc = new Document() > for (int i = 0; i < pages.length; i++) { > doc.add(new Field("text", pages[i], Field.Store.NO, > Field.Index.TOKENIZED)); > doc.add(new Field("text", "$$", Field.Store.NO, > Field.Index.UN_TOKENIZED)); > } UN_TOKENIZED. Nice idea! I will check this

RE: Design questions

2008-02-15 Thread spring
> Why not just use ? Because nearly every analyzer removes it (SimpleAnalyzer, German, Russian, French...) Just tested it with luke in the search dialog. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional comman

Re: Design questions

2008-02-15 Thread Adrian Smith
Hi, I have a similar sitaution. I also considered using $. But for the sake of not running into (potential) problems with Tokenisers, I just defined a string in a config file which for sure is never going to occur in a document and will never be searched for, e.g. dfgjkjrkruigduhfkdgjrugr Cheers

Re: How to pass additional information into Similarity.scorePayload(...)

2008-02-15 Thread Cedric Ho
Hi Paul, Do you mean the following? e.g. to index this: "first second third forth fifth six" originally it would be indexed as: (first,0) (second,1) (third,2) (forth,3) (fifth,4) (six,5) now it will be: (first,0) (second,0) (third,0) (forth,1) (fifth,1) (six,1) Then those Query classes that d