No, I've never used Luke. Is there an easy way to examine my
RAMDirectory
index? I can create the index with no quoted keywords, and when I
search
for a keyword, I get back the expected results (just can't search for a
phrase that has whitespace in it). If I create the index with
phrases in
quotes, then when I search for anything in double quotes, I get back
nothing. If I create the index with everything in quotes, then when I
search for anything by the keyword field, I get nothing, regardless of
whether I use quotes in the query string or not. (I can get results
back
by
searching on other fields.) What do you think?
Philip
Erick Erickson wrote:
>
> OK, I've gotta ask. Have you examined your index with Luke to see if
what
> you *think* is in the index actually *is*???
>
> Erick
>
> On 9/1/06, Philip Brown <[EMAIL PROTECTED]> wrote:
>>
>>
>> Interesting...just ran a test where I put double quotes around
everything
>> (including single keywords) of source text and then ran searches
for a
>> known
>> keyword with and without double quotes -- doesn't find either time.
>>
>>
>> Mark Miller-5 wrote:
>> >
>> > Sorry to hear you're having trouble. You indeed need the double
quotes
>> in
>> > the source text. You will also need them in the query string. Make
sure
>> > they
>> > are in both places. My machine is hosed right now or I would do it
for
>> you
>> > real quick. My guess is that I forgot to mention...no only do you
need
>> to
>> > add the <QUOTED> definiton to the TOKEN section, but below that you
>> will
>> > find the grammer...you need to add <QUOTED> to the grammer. If you
look
>> > how
>> > <NUM> and <APOSTROPHE> are done you will prob see what you
should do.
>> If
>> > not, my machine should be back up tomarrow...
>> >
>> > - Mark
>> >
>> > On 9/1/06, Philip Brown <[EMAIL PROTECTED]> wrote:
>> >>
>> >>
>> >> Well, I tried that, and it doesn't seem to work still. I would be
>> happy
>> >> to
>> >> zip up the new files, so you can see what I'm using -- maybe
you can
>> get
>> >> it
>> >> to work. The first time, I tried building the documents without
>> quotes
>> >> surrounding each phrase. Then, I retried by enclosing every
phrase
>> >> within
>> >> double quotes. Neither seemed to work. When constructing the
query
>> >> string
>> >> for the search, I always added the double quotes (otherwise, it'd
>> think
>> >> it
>> >> was multiple terms). (I didn't even test the underscore and
>> hyphenated
>> >> terms.) I thought Lucene was (sort of by default) set up to
search
>> >> quoted
>> >> phrases. From
http://lucene.apache.org/java/docs/api/index.html -->
A
>> >> Phrase is a group of words surrounded by double quotes such as
"hello
>> >> dolly". So, this should be easy, right? I must be missing
something
>> >> stupid.
>> >>
>> >> Thanks,
>> >>
>> >> Philip
>> >>
>> >>
>> >> Mark Miller-5 wrote:
>> >> >
>> >> > So this will recognize anything in quotes as a single token and
'_'
>> and
>> >> > '-' will not break up words. There may be some repercussions for
the
>> >> NUM
>> >> > token but nothing I'd worry about. maybe you want to use Unicode
for
>> >> '-'
>> >> > and '_' as well...I wouldn't worry about it myself.
>> >> >
>> >> > - Mark
>> >> >
>> >> >
>> >> > TOKEN : { // token patterns
>> >> >
>> >> > // basic word: a sequence of digits & letters
>> >> > <ALPHANUM: (<LETTER>|<DIGIT>|<KOREAN>)+ >
>> >> >
>> >> > | <QUOTED: "\"" (~["\""])+ "\"">
>> >> >
>> >> > // internal apostrophes: O'Reilly, you're, O'Reilly's
>> >> > // use a post-filter to remove possesives
>> >> > | <APOSTROPHE: <ALPHA> ("'" <ALPHA>)+ >
>> >> >
>> >> > // acronyms: U.S.A., I.B.M., etc.
>> >> > // use a post-filter to remove dots
>> >> > | <ACRONYM: <ALPHA> "." (<ALPHA> ".")+ >
>> >> >
>> >> > // company names like AT&T and [EMAIL PROTECTED]
>> >> > | <COMPANY: <ALPHA> ("&"|"@") <ALPHA> >
>> >> >
>> >> > // email addresses
>> >> > | <EMAIL: <ALPHANUM> (("."|"-"|"_") <ALPHANUM>)* "@" <ALPHANUM>
>> >> > (("."|"-") <ALPHANUM>)+ >
>> >> >
>> >> > // hostname
>> >> > | <HOST: <ALPHANUM> ("." <ALPHANUM>)+ >
>> >> >
>> >> > // floating point, serial, model numbers, ip addresses, etc.
>> >> > // every other segment must have at least one digit
>> >> > | <NUM: (<ALPHANUM> <P> <HAS_DIGIT>
>> >> > | <HAS_DIGIT> <P> <ALPHANUM>
>> >> > | <ALPHANUM> (<P> <HAS_DIGIT> <P> <ALPHANUM>)+
>> >> > | <HAS_DIGIT> (<P> <ALPHANUM> <P> <HAS_DIGIT>)+
>> >> > | <ALPHANUM> <P> <HAS_DIGIT> (<P> <ALPHANUM> <P>
>> <HAS_DIGIT>)+
>> >> > | <HAS_DIGIT> <P> <ALPHANUM> (<P> <HAS_DIGIT> <P>
>> <ALPHANUM>)+
>> >> > )
>> >> > >
>> >> > | <#P: ("_"|"-"|"/"|"."|",") >
>> >> > | <#HAS_DIGIT: // at least one digit
>> >> > (<LETTER>|<DIGIT>)*
>> >> > <DIGIT>
>> >> > (<LETTER>|<DIGIT>)*
>> >> > >
>> >> >
>> >> > | < #ALPHA: (<LETTER>)+>
>> >> > | < #LETTER: // unicode letters
>> >> > [
>> >> > "\u0041"-"\u005a",
>> >> > "\u0061"-"\u007a",
>> >> > "\u00c0"-"\u00d6",
>> >> > "\u00d8"-"\u00f6",
>> >> > "\u00f8"-"\u00ff",
>> >> > "\u0100"-"\u1fff",
>> >> > "-", "_"
>> >> > ]
>> >> > >
>> >> >
>> >> >
>> ---------------------------------------------------------------------
>> >> > To unsubscribe, e-mail: [EMAIL PROTECTED]
>> >> > For additional commands, e-mail:
[EMAIL PROTECTED]
>> >> >
>> >> >
>> >> >
>> >>
>> >> --
>> >> View this message in context:
>> >>
>>
http://www.nabble.com/Phrase-search-using-quotes----special-Tokenizer-tf2200760.html#a6106920
>> >> Sent from the Lucene - Java Users forum at Nabble.com.
>> >>
>> >>
>> >>
---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> >> For additional commands, e-mail: [EMAIL PROTECTED]
>> >>
>> >>
>> >
>> >
>>
>> --
>> View this message in context:
>>
http://www.nabble.com/Phrase-search-using-quotes----special-Tokenizer-tf2200760.html#a6107649
>> Sent from the Lucene - Java Users forum at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>>
>
>
--
View this message in context:
http://www.nabble.com/Phrase-search-using-quotes----special-Tokenizer-tf2200760.html#a6109067
Sent from the Lucene - Java Users forum at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
