date:20100323

Re: phrase segmentation plugin in component, analyzer, filter or parser?

2010-03-23 Thread Erik Hatcher



On Mar 24, 2010, at 1:35 AM, Tommy Chheng wrote:

I'm writing an experimental phrase segmentation plugin for solr.

My current plan is to write as a SearchComponent by overriding the  
queryString with the new grouped query.
ex. (university of california irvine 2009) will be re-written to  
"university of calfornia irvine" "2009"



Is the SearchComponent the right class to extend for this type of  
logic?
I picked the component because it was one place where i could get  
access to overwrite the whole query string.


Or is it better design to write it as an analyzer, tokenizer, filter  
or parser plugin?


Seems like a QParserPlugin (and corresponding QParser) are what fit  
best here.  And you may need to have some corresponding analysis  
tricks to ensure things get indexed as your query parser expects for  
search.


Erik

phrase segmentation plugin in component, analyzer, filter or parser?

2010-03-23 Thread Tommy Chheng


 I'm writing an experimental phrase segmentation plugin for solr.

My current plan is to write as a SearchComponent by overriding the 
queryString with the new grouped query.
ex. (university of california irvine 2009) will be re-written to 
"university of calfornia irvine" "2009"



Is the SearchComponent the right class to extend for this type of logic?
I picked the component because it was one place where i could get access 
to overwrite the whole query string.


Or is it better design to write it as an analyzer, tokenizer, filter or 
parser plugin?



--
Tommy Chheng
Programmer and UC Irvine Graduate Student
Twitter @tommychheng
http://tommy.chheng.com

Re: dismax and q.op

2010-03-23 Thread Mark Fletcher

Hi Hoss,

Thankyou so much for your time.

Regarding the last one I myself got confused when I posed the question. I
got it after your reply. I think I was actually looking for some thing like
the debugQuery="on" option, which I found later.

Best Regards,
Mark.

On Tue, Mar 23, 2010 at 6:56 PM, Chris Hostetter
wrote:

>
> :  *I haven't mentioned value for mm*
>...
> : My result:- No results; but each of the terms individually gave me
> results!
>
>
> http://wiki.apache.org/solr/DisMaxRequestHandler#mm_.28Minimum_.27Should.27_Match.29
>
>"The default value is 100% (all clauses must match)"
>
> : 2. Does the default operator specified in schema.xml take effect when we
> use
> : dismax also or is it only for the *standard* request handler. If it has
> an
>
> dismax doesn't look at the default operator, or q.op.
>
> : 3. How does q.alt and q difer in behavior in the above case. I found
> q.alt
> : to be giving me the results which I got when I used the standard RH also.
> : Hence used it.
>
> q.alt is used if and only if there is no q param (or hte q param is blank)
> ... the number of patches "q" gets, or the value of "mm" make no
> differnce.
>
> : 4. When I make a change to the dismax set up I have in solrconfig.xml I
> : believe i just have to bounce the SOLR server.Do i need to re-index again
> : for the change to take effect
>
> no ... changes to "query" time options like your SearchHandler configs
> don't require reindexing .. changes to your schema.xml *may* requre
> reindexing.
>
> : 5. If I use the dismax how do I see the ANALYSIS feature on the admin
> : console other wise used for *standard* RH.
>
> I'm afraid i don't understand this question ... analysis.jsp just shows
> you the index and query time analysis that is performed when certain
> fields are used -- it dosen't know/care about your choice of parser ... it
> knows nothing about query parser syntax.
>
>
>
> -Hoss
>
>

Re: HTTP Status 500 - null java.lang.IllegalArgumentException at java.nio.Buffer.limit(Buffer.java:249)

2010-03-23 Thread Lance Norskog

That area of the Lucene code throws NullPEs and ArrayIndex bugs, but
they are all caused by corrupt indexes. They should be caught and
wrapped.

On Tue, Mar 23, 2010 at 4:33 PM, Chris Hostetter
 wrote:
>
> : I am doing a really simple query on my index (it's running in tomcat):
> :
> : http://host:8080/solr_er_07_09/select/?q=hash_id:123456
>        ...
>
> details please ...
>
>    http://wiki.apache.org/solr/UsingMailingLists
>
> ... what version of solr? lucene? tomcat?
>
> : I built the index on a different machine than the one I am doing the
>
> ...ditto for that machine.
>
> are you sure hte md5 checksums match for both copies of the index (ie: did
> it get corrupted when you copied it)
>
> what does CheckIndex say about hte index?
>
> : query on though the configuration is exactly the same. I can do the same
> : query using solrj (I have an app doing that) and it works fine.
>
> that seems highly bizzare ... are you certain it's the exact same query?
> what does the tomcat log say about hte two requests?
>
>
>
> -Hoss
>
>



-- 
Lance Norskog
goks...@gmail.com

Re: Impossible Boost Query?

2010-03-23 Thread Lance Norskog

Also, there is a 'random' type which generates random numbers. This
might help you also.

On Tue, Mar 23, 2010 at 7:18 PM, Lance Norskog  wrote:
> At this point (and for almost 3 years :) field collapsing is a source
> patch. You have to check out the Solr trunk from the Apache subversion
> server, apply the patch with the 'patch' command, and build the new
> Solr with 'ant'.
>
> On Tue, Mar 23, 2010 at 4:13 PM, blargy  wrote:
>>
>> Thanks but Im not quite show on how to apply the patch. I just use the
>> packaged solr-1.4.0.war in my deployment (no compiling, etc). Is there a way
>> I can patch the war file?
>>
>> Any instructions would be greatly appreciated. Thanks
>>
>>
>> Otis Gospodnetic wrote:
>>>
>>> You'd likely want to get the latest patch and trunk and try applying.
>>>
>>> Otis
>>> 
>>> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
>>> Hadoop ecosystem search :: http://search-hadoop.com/
>>>
>>>
>>>
>>> - Original Message 
 From: blargy 
 To: solr-user@lucene.apache.org
 Sent: Tue, March 23, 2010 6:10:22 PM
 Subject: Re: Impossible Boost Query?


>>> Maybe a better question is... how can I install this and will it work
 with
>>> 1.4?
>>>
>>> Thanks
>>>
>>>
>>> blargy wrote:

 Possibly.
 How can I install this as a contrib or do I need to actually
 perform the
 patch?


 Otis Gospodnetic wrote:
>

> Would Field Collapsing from SOLR-236 do the job for
 you?
>
> Otis
> 
> Sematext ::
 href="http://sematext.com/"; target=_blank >http://sematext.com/ :: Solr -
 Lucene - Nutch
> Hadoop ecosystem search ::
 href="http://search-hadoop.com/"; target=_blank
 >http://search-hadoop.com/
>
>
>

> - Original Message 
>> From: blargy <
 ymailto="mailto:zman...@hotmail.com";
 href="mailto:zman...@hotmail.com";>zman...@hotmail.com>
>>
 To:
 href="mailto:solr-user@lucene.apache.org";>solr-user@lucene.apache.org
>>
 Sent: Tue, March 23, 2010 2:39:48 PM
>> Subject: Impossible Boost
 Query?
>>
>>
> I was wondering if this is
 even possible. I'll try to explain what I'm
>> trying
>
 to do to the best of my ability.
>
> Ok, so our site has a
 bunch
>> of products that are sold by any number of
>
 sellers. Currently when I search
>> for some product I get back
 all products
> matching that search term but the
>>
 problem is there may be multiple products
> sold by the same seller
 that are
>> all closely related, therefore their
 scores
> are related. So basically the
>> search ends up
 with results that are all
> closely clumped together by the same

>> seller but I would much rather prefer
> to distribute
 these results across
>> sellers (given each seller a fair shot
 to
> sell their goods).
>
> Is there

>> any way to add some boost query for example that will
 start
> weighing products
>> lower when their seller has
 already been listed a few
> times. For example,
>> right
 now I have
>
> Product foo by Seller A
> Product
 foo by Seller
>> A
> Product foo by Seller A
>
 Product foo by Seller B
> Product foo by Seller
>>
 B
> Product foo by Seller B
> Product foo by Seller
 C
> Product foo by Seller
>> C
> Product foo
 by Seller C
>
> where each result is very close in score. I

>> would like something like this
>
> Product
 foo by Seller A
> Product foo by
>> Seller B
>
 Product foo by Seller C
> Product foo by Seller A
> Product
 foo by
>> Seller B
> Product foo by Seller C
>
 
>
> basically distributing the
>>
 results over the sellers. Is something like this
> possible? I don't
 care if
>> the solution involves a boost query or not. I
 just
> want some way to
>> distribute closely related
 documents.
>
> Thanks!!!
> --
> View
 this
>> message in context:
>> href="
 href="http://old.nabble.com/Impossible-Boost-Query--tp28005354p28005354.html";
 target=_blank
 >http://old.nabble.com/Impossible-Boost-Query--tp28005354p28005354.html";

>> target=_blank
>> >
 href="http://old.nabble.com/Impossible-Boost-Query--tp28005354p28005354.html";
 target=_blank
 >http://old.nabble.com/Impossible-Boost-Query--tp28005354p28005354.html
>
 Sent
>> from the Solr - User mailing list archive at
 Nabble.com.
>
>


>>>
>>> --
>>> View this
 message in context:
 href="http://old.nabble.com/Impossible-Boost-Query--tp28005354p28007880.html";
 target=_blank
 >http://old.nabble.com/Impossible-Boost-Query--tp28005354p28007880.html
>>> Sent
 from the Solr - User

Re: Impossible Boost Query?

2010-03-23 Thread Lance Norskog

At this point (and for almost 3 years :) field collapsing is a source
patch. You have to check out the Solr trunk from the Apache subversion
server, apply the patch with the 'patch' command, and build the new
Solr with 'ant'.

On Tue, Mar 23, 2010 at 4:13 PM, blargy  wrote:
>
> Thanks but Im not quite show on how to apply the patch. I just use the
> packaged solr-1.4.0.war in my deployment (no compiling, etc). Is there a way
> I can patch the war file?
>
> Any instructions would be greatly appreciated. Thanks
>
>
> Otis Gospodnetic wrote:
>>
>> You'd likely want to get the latest patch and trunk and try applying.
>>
>> Otis
>> 
>> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
>> Hadoop ecosystem search :: http://search-hadoop.com/
>>
>>
>>
>> - Original Message 
>>> From: blargy 
>>> To: solr-user@lucene.apache.org
>>> Sent: Tue, March 23, 2010 6:10:22 PM
>>> Subject: Re: Impossible Boost Query?
>>>
>>>
>> Maybe a better question is... how can I install this and will it work
>>> with
>> 1.4?
>>
>> Thanks
>>
>>
>> blargy wrote:
>>>
>>> Possibly.
>>> How can I install this as a contrib or do I need to actually
>>> perform the
>>> patch?
>>>
>>>
>>> Otis Gospodnetic wrote:

>>>
 Would Field Collapsing from SOLR-236 do the job for
>>> you?

 Otis
 
 Sematext ::
>>> href="http://sematext.com/"; target=_blank >http://sematext.com/ :: Solr -
>>> Lucene - Nutch
 Hadoop ecosystem search ::
>>> href="http://search-hadoop.com/"; target=_blank
>>> >http://search-hadoop.com/



>>>
 - Original Message 
> From: blargy <
>>> ymailto="mailto:zman...@hotmail.com";
>>> href="mailto:zman...@hotmail.com";>zman...@hotmail.com>
>
>>> To:
>>> href="mailto:solr-user@lucene.apache.org";>solr-user@lucene.apache.org
>
>>> Sent: Tue, March 23, 2010 2:39:48 PM
> Subject: Impossible Boost
>>> Query?
>
>
 I was wondering if this is
>>> even possible. I'll try to explain what I'm
> trying

>>> to do to the best of my ability.

 Ok, so our site has a
>>> bunch
> of products that are sold by any number of

>>> sellers. Currently when I search
> for some product I get back
>>> all products
 matching that search term but the
>
>>> problem is there may be multiple products
 sold by the same seller
>>> that are
> all closely related, therefore their
>>> scores
 are related. So basically the
> search ends up
>>> with results that are all
 closely clumped together by the same
>>>
> seller but I would much rather prefer
 to distribute
>>> these results across
> sellers (given each seller a fair shot
>>> to
 sell their goods).

 Is there
>>>
> any way to add some boost query for example that will
>>> start
 weighing products
> lower when their seller has
>>> already been listed a few
 times. For example,
> right
>>> now I have

 Product foo by Seller A
 Product
>>> foo by Seller
> A
 Product foo by Seller A

>>> Product foo by Seller B
 Product foo by Seller
>
>>> B
 Product foo by Seller B
 Product foo by Seller
>>> C
 Product foo by Seller
> C
 Product foo
>>> by Seller C

 where each result is very close in score. I
>>>
> would like something like this

 Product
>>> foo by Seller A
 Product foo by
> Seller B

>>> Product foo by Seller C
 Product foo by Seller A
 Product
>>> foo by
> Seller B
 Product foo by Seller C

>>> 

 basically distributing the
>
>>> results over the sellers. Is something like this
 possible? I don't
>>> care if
> the solution involves a boost query or not. I
>>> just
 want some way to
> distribute closely related
>>> documents.

 Thanks!!!
 --
 View
>>> this
> message in context:
> href="
>>> href="http://old.nabble.com/Impossible-Boost-Query--tp28005354p28005354.html";
>>> target=_blank
>>> >http://old.nabble.com/Impossible-Boost-Query--tp28005354p28005354.html";
>>>
> target=_blank
> >
>>> href="http://old.nabble.com/Impossible-Boost-Query--tp28005354p28005354.html";
>>> target=_blank
>>> >http://old.nabble.com/Impossible-Boost-Query--tp28005354p28005354.html

>>> Sent
> from the Solr - User mailing list archive at
>>> Nabble.com.


>>>
>>>
>>
>> --
>> View this
>>> message in context:
>>> href="http://old.nabble.com/Impossible-Boost-Query--tp28005354p28007880.html";
>>> target=_blank
>>> >http://old.nabble.com/Impossible-Boost-Query--tp28005354p28007880.html
>> Sent
>>> from the Solr - User mailing list archive at Nabble.com.
>>
>>
>
> --
> View this message in context: 
> http://old.nabble.com/Impossible-Boost-Query--tp28005354p28008495.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>



-- 
Lance Norskog
goks...@gmail.com

Re: SOLR-1316 How To Implement this autosuggest component ???

2010-03-23 Thread Lance Norskog

You need 'ant' to do builds.  At the top level, do:
ant clean
ant example

These will build everything and set up the example/ directory. After that, run:
ant test-core

to run all of the unit tests and make sure that the build works. If
the autosuggest patch has a test, this will check that the patch went
in correctly.

Lance

On Tue, Mar 23, 2010 at 7:42 AM, stocki  wrote:
>
> okay,
> i do this..
>
> but one file are not right updatet 
> Index: trunk/src/java/org/apache/solr/util/HighFrequencyDictionary.java
> (from the suggest.patch)
>
> i checkout it from eclipse, apply patch, make an new solr.war ... its the
> right way ??
> i thought that is making a war i didnt need to make an build.
>
> how do i make an build ?
>
>
>
>
> Alexey-34 wrote:
>>
>>> Error loading class 'org.apache.solr.spelling.suggest.Suggester'
>> Are you sure you applied the patch correctly?
>> See http://wiki.apache.org/solr/HowToContribute#Working_With_Patches
>>
>> Checkout Solr trunk source code (
>> http://svn.apache.org/repos/asf/lucene/solr/trunk ), apply patch,
>> verify that everything went smoothly, build solr and use built version
>> for your tests.
>>
>> On Mon, Mar 22, 2010 at 9:42 PM, stocki  wrote:
>>>
>>> i patch an nightly build from solr.
>>> patch runs, classes are in the correct folder, but when i replace
>>> spellcheck
>>> with this spellchecl like in the comments, solr cannot find the classes
>>> =(
>>>
>>> 
>>>    
>>>      suggest
>>>      >> name="classname">org.apache.solr.spelling.suggest.Suggester
>>>      >> name="lookupImpl">org.apache.solr.spelling.suggest.jaspell.JaspellLookup
>>>      text
>>>      american-english
>>>    
>>>  
>>>
>>>
>>> --> SCHWERWIEGEND: org.apache.solr.common.SolrException: Error loading
>>> class
>>> 'org.ap
>>> ache.solr.spelling.suggest.Suggester'
>>>
>>>
>>> why is it so ??  i think no one has so many trouble to run a patch
>>> like
>>> me =( :D
>>>
>>>
>>> Andrzej Bialecki wrote:

 On 2010-03-19 13:03, stocki wrote:
>
> hello..
>
> i try to implement autosuggest component from these link:
> http://issues.apache.org/jira/browse/SOLR-1316
>
> but i have no idea how to do this !?? can anyone get me some tipps ?

 Please follow the instructions outlined in the JIRA issue, in the
 comment that shows fragments of XML config files.


 --
 Best regards,
 Andrzej Bialecki     <><
   ___. ___ ___ ___ _ _   __
 [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
 ___|||__||  \|  ||  |  Embedded Unix, System Integration
 http://www.sigram.com  Contact: info at sigram dot com



>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/SOLR-1316-How-To-Implement-this-autosuggest-component-tp27950949p27990809.html
>>> Sent from the Solr - User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>
> --
> View this message in context: 
> http://old.nabble.com/SOLR-1316-How-To-Implement-this-patch-autoComplete-tp27950949p28001938.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>



-- 
Lance Norskog
goks...@gmail.com

SOLR-236 patch with version 1.4

2010-03-23 Thread blargy

Is the field collapsing patch (236) not compatible with Solr 1.4?

$ patch -p0 -i ~/Desktop/SOLR-236.patch
patching file src/test/test-files/solr/conf/solrconfig-fieldcollapse.xml
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/DocumentGroupCountCollapseCollectorFactory.java
patching file
src/java/org/apache/solr/search/fieldcollapse/CollapseGroup.java
patching file
src/java/org/apache/solr/search/fieldcollapse/AdjacentDocumentCollapser.java
patching file src/java/org/apache/solr/search/DocSetAwareCollector.java
patching file src/java/org/apache/solr/search/SolrIndexSearcher.java
Hunk #1 FAILED at 17.
Hunk #2 FAILED at 530.
Hunk #3 FAILED at 586.
Hunk #4 FAILED at 610.
Hunk #5 FAILED at 663.
Hunk #6 FAILED at 705.
Hunk #7 FAILED at 716.
Hunk #8 FAILED at 740.
Hunk #9 FAILED at 1255.
9 out of 9 hunks FAILED -- saving rejects to file
src/java/org/apache/solr/search/SolrIndexSearcher.java.rej
patching file
src/java/org/apache/solr/handler/component/CollapseComponent.java
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/CollapseCollectorFactory.java
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/aggregate/AggregateFunction.java
patching file
src/test/org/apache/solr/search/fieldcollapse/NonAdjacentDocumentCollapserTest.java
patching file src/java/org/apache/solr/util/DocSetScoreCollector.java
patching file
src/java/org/apache/solr/search/fieldcollapse/AbstractDocumentCollapser.java
patching file
src/java/org/apache/solr/search/fieldcollapse/util/Counter.java
patching file
src/java/org/apache/solr/search/fieldcollapse/DocumentCollapser.java
patching file
src/solrj/org/apache/solr/client/solrj/response/FieldCollapseResponse.java
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/FieldValueCountCollapseCollectorFactory.java
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/CollapseCollector.java
patching file
src/test/org/apache/solr/search/fieldcollapse/DistributedFieldCollapsingIntegrationTest.java
patching file
src/test/org/apache/solr/client/solrj/response/FieldCollapseResponseTest.java
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/aggregate/MaxFunction.java
patching file src/test/test-files/solr/conf/solrconfig.xml
Hunk #1 FAILED at 396.
Hunk #2 FAILED at 418.
2 out of 2 hunks FAILED -- saving rejects to file
src/test/test-files/solr/conf/solrconfig.xml.rej
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/CollapseContext.java
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/aggregate/MinFunction.java
patching file
src/solrj/org/apache/solr/client/solrj/response/QueryResponse.java
Hunk #1 FAILED at 17.
Hunk #2 FAILED at 42.
Hunk #3 FAILED at 58.
Hunk #4 FAILED at 125.
Hunk #5 FAILED at 298.
5 out of 5 hunks FAILED -- saving rejects to file
src/solrj/org/apache/solr/client/solrj/response/QueryResponse.java.rej
patching file src/test/test-files/fieldcollapse/testResponse.xml
patching file
src/java/org/apache/solr/search/fieldcollapse/NonAdjacentDocumentCollapser.java
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/AbstractCollapseCollector.java
patching file src/java/org/apache/solr/handler/component/QueryComponent.java
Hunk #1 FAILED at 522.
1 out of 1 hunk FAILED -- saving rejects to file
src/java/org/apache/solr/handler/component/QueryComponent.java.rej
patching file
src/java/org/apache/solr/search/fieldcollapse/DocumentCollapseResult.java
patching file
src/test/org/apache/solr/handler/component/CollapseComponentTest.java
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/DocumentFieldsCollapseCollectorFactory.java
patching file src/test/test-files/solr/conf/schema-fieldcollapse.xml
patching file src/common/org/apache/solr/common/params/CollapseParams.java
patching file src/solrj/org/apache/solr/client/solrj/SolrQuery.java
Hunk #1 FAILED at 17.
Hunk #2 FAILED at 50.
Hunk #3 FAILED at 76.
Hunk #4 FAILED at 148.
Hunk #5 FAILED at 197.
Hunk #6 FAILED at 665.
Hunk #7 FAILED at 721.
7 out of 7 hunks FAILED -- saving rejects to file
src/solrj/org/apache/solr/client/solrj/SolrQuery.java.rej
patching file
src/test/org/apache/solr/search/fieldcollapse/AdjacentCollapserTest.java
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/AggregateCollapseCollectorFactory.java
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/aggregate/SumFunction.java
patching file src/java/org/apache/solr/search/DocSetHitCollector.java
Hunk #1 FAILED at 17.
Hunk #2 FAILED at 28.
2 out of 2 hunks FAILED -- saving rejects to file
src/java/org/apache/solr/search/DocSetHitCollector.java.rej
patching file
src/java/org/apache/solr/search/fieldcollapse/collector/aggregate/AverageFunction.java
patching file
src/test/org/apache/solr/search/fieldcollapse/FieldCollapsingIntegrationTest.java

71 matches

Mail list logo