date:20130912

Re: exceeded limit of maxWarmingSearchers

2013-09-12 Thread Erick Erickson

I really think this is the wrong approach.

bq: We do a commit on every update, but updates are very infrequent

I doubt this is actually true. You may think it is, but you just don't get
more than 8 warming searchers in the situation you describe. Fix the
_real_ problem here.

Do what Hoss said. Look at your logs and examine the commits
that happen just before you get this message. I pretty much guarantee
that you will find a flurry of them in quick succession. Very quick
succession.

I claim you can get around this problem entirely by

1> setting your autocommit (hard, openSearcher=true) to
10 seconds. Which, btw, is very very low. I'd go with a minute
or more myself, and also configure soft commits if you're
on Solr 4 and really really care about latency.

2> never committing from the client.

The reason I'm adamant about this is what you're doing will come
back to bite you in the future if you don't fix the problem now. As you
get more and more documents in the system, your 8 warming
searchers will try to warm 8 versions of the caches you've configured
in solrconfig.xml. If your index is very large, you'll hit OOM errors.
Then you'll have to figure out why all over again.

Really, with 37ms warming, you have a pathological situation that
you need to understand and fix.

Best,
Erick

On Thu, Sep 12, 2013 at 4:18 PM, gfbj  wrote:

> I ended up having to do a mathematical increase of the delay
>
> 
>
> because the indexing eventually would outstrip the static value I set and
> crash the maxWarmingSearchers.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/exceeded-limit-of-maxWarmingSearchers-tp489803p4089699.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Different Responses for 4.4 and 3.5 solr index

2013-09-12 Thread Kuchekar

Hi,

Any updates on this?. Is ranking computation dependent on the 'maxDoc'
value in the solr? Is this happening due to changing value of 'maxDoc'
value after each optimization. As in, in solr 4.4 every time optimization
is ran, the 'maxDoc' value is reset, where as this is not the case in solr
3.5.

Looking forward for the reply.

Thanks.
Kuchekar, Nilesh


On Wed, Aug 28, 2013 at 3:32 PM, Michael Sokolov <
msoko...@safaribooksonline.com> wrote:

> We've been seeing changes in our rankings as well.  I don't have a
> definite answer yet, since we're waiting on an index rebuild, but our
> current working theory is that the change to default omitNorms="true" for
> primitive types may have had an effect, possibly due to follow on
> confusion: our developers may have omitted norms from some other fields
> they shouldn't have?
>
> -Mike
>
>
> On 08/26/2013 09:46 AM, Stefan Matheis wrote:
>
>> Did you check the scoring? (use fl=*,score to retrieve it) ..
>> additionally debugQuery=true might provide more information about how the
>> score was calculated.
>>
>> - Stefan
>>
>>
>> On Monday, August 26, 2013 at 12:46 AM, Kuchekar wrote:
>>
>>  Hi,
>>> The response from 4.4 and 3.5 in the current scenario differs in the
>>> sequence in which results are given us back.
>>>
>>> For example :
>>>
>>> Response from 3.5 solr is : id:A, id:B, id:C, id:D ...
>>> Response from 4.4 solr is : id C, id:A, id:D, id:B...
>>>
>>> Looking forward your reply.
>>>
>>> Thanks.
>>> Kuchekar, Nilesh
>>>
>>>
>>> On Sun, Aug 25, 2013 at 11:32 AM, Stefan Matheis
>>> >> (mailto:matheis.stefan@gmail.**com
>>> )>wrote:
>>>
>>>  Kuchekar (hope that's your first name?)

 you didn't tell us .. how they differ? do you get an actual error? or
 does
 the result contain documents you didn't expect? or the other way round,
 that some are missing you'd expect to be there?

 - Stefan


 On Sunday, August 25, 2013 at 4:43 PM, Kuchekar wrote:

  Hi,
>
> We get different response when we query 4.4 and 3.5 solr using same
> query params.
>
> My query param are as following :
>
> facet=true
> &facet.mincount=1
> &facet.limit=25
>
>  &qf=content^0.0+p_last_name^**500.0+p_first_name^50.0+**
 strong_topic^0.0+first_author_**topic^0.0+last_author_topic^0.**
 0+title_topic^0.0

> &wt=javabin
> &version=2
> &rows=10
> &f.affiliation_org.facet.**limit=150
> &fl=p_id,p_first_name,p_last_**name
> &start=0
> &q=Apple
> &facet.field=affiliation_org
> &fq=table:profile
> &fq=num_content:[*+TO+1500]
> &fq=name:"Apple"
>
> The content in both (solr 4.4 and solr 3.5) are same.
>
> The solrconfig.xml from 3.5 an 4.4 are similarly constructed.
>
> Is there something I am missing that might have been changed in 4.4,
>
 which

> might be causing this issue. ?. The "qf" params looks same.
>
> Looking forward for your reply.
>
> Thanks.
> Kuchekar, Nilesh
>
>

>>>
>>>
>>
>>
>

Solr 4.5 spatial search - distance and score

2013-09-12 Thread Weber

I'm trying to get score by using a custom boost and also get the distance. I
found David's code* to get it using "Intersects", which I want to replace by
{!geofilt} or geodist()

*David's code: https://issues.apache.org/jira/browse/SOLR-4255

He told me geodist() will be available again for this kind of field, which
is a geohash type.

Then, I'd like to know how it can be done today on 4.4 with {!geofilt} and
how it will be done on 4.5 using geodist()

Thanks in advance.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-4-5-spatial-search-distance-and-score-tp4089706.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Get the commit time of a document in Solr

2013-09-12 Thread Otis Gospodnetic

Solr admin exposes time of last commit. You can use that.

Otis
Solr & ElasticSearch Support
http://sematext.com/
On Sep 12, 2013 3:22 PM, "phanichaitanya"  wrote:

> Apologies again. But here is another try :
>
> I want to make sure that documents that are indexed are committed in say an
> hour. I agree that if you pass commitWithIn params and the like will make
> sure of that based on the time configurations we set. But, I want to make
> sure that the document is really committed within whatever time we set
> using
> commitWithIn.
>
> It's a question asking for proof that Solr commits within that time if we
> add commitWithIn parameter to the configuration.
>
> That is about commitWithIn parameter option that you suggested.
>
> Now is there a way to explicitly get all the documents that are committed
> when a hard commit request is issued ? This might not make sense but we are
> pondered with that question.
>
>
>
> -
> Phani Chaitanya
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Get-the-commit-time-of-a-document-in-Solr-tp4089624p4089687.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Stop filter changes in Solr >= 4.4

2013-09-12 Thread Christopher Condit

While attempting to upgrade from Solr 4.3.0 to Solr 4.4.0 I ran into
this exception:

java.lang.IllegalArgumentException: enablePositionIncrements=false is
not supported anymore as of Lucene 4.4 as it can create broken token
streams

which led me to https://issues.apache.org/jira/browse/LUCENE-4963. I
need to be able to match queries irrespective of intervening stopwords
(which used to work with enablePositionIncrements="true"). For
instance: "foo of the bar" would find documents matching "foo bar",
"foo of bar", and "foo of the bar". With this option deprecated in
4.4.0 I'm not clear on how to maintain the same functionality.

The package javadoc adds:

If the selected analyzer filters the stop words "is" and "the", then
for a document containing the string "blue is the sky", only the
tokens "blue", "sky" are indexed, with position("sky") = 3 +
position("blue"). Now, a phrase query "blue is the sky" would find
that document, because the same analyzer filters the same stop words
from that query. But the phrase query "blue sky" would not find that
document because the position increment between "blue" and "sky" is
only 1.

If this behavior does not fit the application needs, the query parser
needs to be configured to not take position increments into account
when generating phrase queries.

But there's no mention of how to actually configure the query parser
to do this. Does anyone know how to deal with this issue as Solr moves
toward 5.0?

53 matches

Mail list logo