RE: SOLR Cloud Rebuild core

[Branham, Jeremy [HR]]

Perfect.
Rather than rolling back a single transaction, we could just rollback 
everything since the last commit.

Thanks for the insight.


Jeremy Branham


-Original Message-
From: Walter Underwood [mailto:wun...@wunderwood.org]
Sent: Saturday, June 14, 2014 3:03 PM
To: solr-user@lucene.apache.org
Subject: Re: SOLR Cloud Rebuild core

Exactly. Do one commit at the end. I do this for indexes with 4 million or more 
documents. Works fine.

I don't know of a way to flush the queued add-document commands. Solr does not 
have the concept of a single transaction that can be dropped.

wunder

On Jun 14, 2014, at 12:57 PM, "Branham, Jeremy [HR]" 
 wrote:

> Thanks Walter -
>
> We have a few different use cases where there is no good way [currently] to 
> uniquely identify a document.
> We also have some cores where real-time freshness is key.
> I'd rather move to cloud than have 2 different instances.
>
> Since some cores will need to have all documents deleted during the rebuild, 
> there a way to do complete rebuild without disturbing queries to the core?
> Maybe postponing the commit to the end of the rebuild process?
>
> I'm thinking the commit would then delete all existing document add all the 
> new documents.
> Is this flawed?
>
> Assuming that would work - if the reindexing fails could we bail on the 
> commit and still have a sane core?
>
> Jeremy Branham
>
> -Original Message-
> From: Walter Underwood [mailto:wun...@wunderwood.org]
> Sent: Saturday, June 14, 2014 2:44 PM
> To: solr-user@lucene.apache.org
> Subject: Re: SOLR Cloud Rebuild core
>
> If you don't need near real-time index freshness, why are you implementing 
> Solr Cloud? It is harder to set up and adds functionality you are not using. 
> Solr Cloud is designed for a fully live index, not offline indexing.
>
> Also, I don't understand why people do this complicated offline build and 
> swapping stuff. That is precisely what replication already does for you and 
> it is built-in. You build an index on one system and swap it in over the 
> network.
>
> wunder
>
> On Jun 14, 2014, at 12:29 PM, "Branham, Jeremy [HR]" 
>  wrote:
>
>> We are looking to move from legacy master/slave configuration to the cloud 
>> configuration.
>>
>> In the past we have handled rebuilding cores by using a 'live' core and a 
>> core for performing the rebuild on.
>> When a rebuild is complete, we swap the rebuilt core with the live core.
>>
>> Is this still a good way to do offline rebuilding when using cloud?
>>
>> Full re-indexing on our largest index only takes 25 min.
>>
>> Thanks!
>>
>>
>> Jeremy Branham
>>
>
>
>
>
>
> 
>
> This e-mail may contain Sprint proprietary information intended for the sole 
> use of the recipient(s). Any use by others is prohibited. If you are not the 
> intended recipient, please contact the sender and delete all copies of the 
> message.
>

--
Walter Underwood
wun...@wunderwood.org






This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

RE: SOLR Cloud Rebuild core

[Branham, Jeremy [HR]]

Thanks Walter -

We have a few different use cases where there is no good way [currently] to 
uniquely identify a document.
We also have some cores where real-time freshness is key.
I'd rather move to cloud than have 2 different instances.

Since some cores will need to have all documents deleted during the rebuild, 
there a way to do complete rebuild without disturbing queries to the core?
Maybe postponing the commit to the end of the rebuild process?

I'm thinking the commit would then delete all existing document add all the new 
documents.
Is this flawed?

Assuming that would work - if the reindexing fails could we bail on the commit 
and still have a sane core?

Jeremy Branham

-Original Message-
From: Walter Underwood [mailto:wun...@wunderwood.org]
Sent: Saturday, June 14, 2014 2:44 PM
To: solr-user@lucene.apache.org
Subject: Re: SOLR Cloud Rebuild core

If you don't need near real-time index freshness, why are you implementing Solr 
Cloud? It is harder to set up and adds functionality you are not using. Solr 
Cloud is designed for a fully live index, not offline indexing.

Also, I don't understand why people do this complicated offline build and 
swapping stuff. That is precisely what replication already does for you and it 
is built-in. You build an index on one system and swap it in over the network.

wunder

On Jun 14, 2014, at 12:29 PM, "Branham, Jeremy [HR]" 
 wrote:

> We are looking to move from legacy master/slave configuration to the cloud 
> configuration.
>
> In the past we have handled rebuilding cores by using a 'live' core and a 
> core for performing the rebuild on.
> When a rebuild is complete, we swap the rebuilt core with the live core.
>
> Is this still a good way to do offline rebuilding when using cloud?
>
> Full re-indexing on our largest index only takes 25 min.
>
> Thanks!
>
>
> Jeremy Branham
>







This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

SOLR Cloud Rebuild core

[Branham, Jeremy [HR]]

We are looking to move from legacy master/slave configuration to the cloud 
configuration.

In the past we have handled rebuilding cores by using a 'live' core and a core 
for performing the rebuild on.
When a rebuild is complete, we swap the rebuilt core with the live core.

Is this still a good way to do offline rebuilding when using cloud?

Full re-indexing on our largest index only takes 25 min.

Thanks!


Jeremy Branham




This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

RE: suspect SOLR query from D029 (SOLR master)

[Branham, Jeremy [HR]]

 121 query terms, which shouldn't be so bad.

But... maybe the Lucene FST for your synonym list is huge. Someone with deeper 
Lucene knowledge would have to address that.

-- Jack Krupansky

-Original Message-
From: Branham, Jeremy [HR]
Sent: Tuesday, June 3, 2014 3:57 AM
To: solr-user@lucene.apache.org
Cc: Worley, Chris [HR]
Subject: RE: suspect SOLR query from D029 (SOLR master)

Evidently I didn't understand enough about the synonym filter.
I'm not sure anyone would be able to determine the impact based on the example 
queries below.

However I'm curious what the best practice is for synonyms.

We have 179 lines of synonyms each with 2 - 6 synonyms per line [all expanded].

The query that hurt us had 11 terms in the search string that matched a synonym 
line with 11 values.
How does this work internally?
Does the query parser create 11^11 search terms that would be parsed?
Should we be looking closer at stemming and tokenization rather than creating 
this list of synonyms?

Just looking for some clarity, or any best practice for use cases that might 
call for synonyms.


Thank you.

Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Monday, June 02, 2014 9:49 PM
To: solr-user@lucene.apache.org
Cc: Worley, Chris [HR]
Subject: RE: suspect SOLR query from D029 (SOLR master)

These are the typical queries we are using.
I'm curious if any of these parameters could be causing issues when using
synonyms.



?shards=myserver1.com:8080/svc/solr/wdsc,myserver1.com:8080/svc/solr/kms&sort=score
desc&q=(keyword:(this is a test) OR titleSearch:(this is a test) AND
(doctype:("Device-Product")^1.1 OR (doctype:("LaunchPacks") OR
doctype:("MenuMaps") OR doctype:("HowToSetupGuide") OR
doctype:("ServiceAdvisory") OR doctype:("TroubleshootingGuide") OR
doctype:("FAQ") OR
doctype:("DeviceManual"&defType=edismax&fq=device_id:("Motorola Moto X
\(XT1056\)")&fq=-doctype:Simulator
AND -doctype:Scenario&bq=doctype:LaunchPacks^1.2 OR doctype:MenuMaps^1.2 OR
doctype:HowToSetupGuide OR doctype:ServiceAdvisory^1.2 OR
doctype:TroubleshootingGuide^1.4 OR doctype:FAQ^2.0 OR doctype:DeviceManual
OR (doctype:Device-Product AND titleSearch:(Programming \-))^15 OR
(doctype:Device-Product AND titleSearch:("Solution Matrix" \-))^5 OR
(doctype:Device-Product AND titleSearch:("Fact Sheet" \-))^4 OR
(doctype:Device-Product AND titleSearch:("User Guide"
\-))^3&fl=id,title,device_id,url,doctype,titleSearch,score&mm=1&start=0&rows=10


?sort=score desc,hits30day
desc&fq=profile:"YY"&fq=channel:ASPC_CustManaCM&q=this is a
test&mm=0&start=0&rows=10


?fq=device_id:("Motorola Moto X \(XT1056\)")&q=(keyword:(this is a test) OR
titleSearch:(this is a test))&start=0&rows=10


?sort=score desc&fq=doctype:ProductKnownIssue&fq=(technology:3G 1900/800,
LTE both) &q=(text:Motorola Moto X (XT1056)^1000 OR text:[* TO *]) AND
(titleSearch:(this is a test)^2 OR text:(this is a
test)^1.5)&start=0&rows=10


?sort=score desc&q=((titleSearch:(this is a test)^2 OR text:(this is a
test)) AND ((device_id:"Motorola Moto X \(XT1056\)" AND category:4247)^20 OR
(category:3676 AND profile:"YY")))&fq=doctype:FlowDoc&fq=(-voice_network:[*
TO *] AND *:*) OR voice_network:3G 1900/800, LTE&start=0&rows=10


?fq=doctype:Scenario&q=device_id:"Motorola Moto X (XT1056)" AND
(titleSearch:(this is a test)^2 OR text:(this is a
test)^1.5)&start=0&rows=10



Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Monday, June 02, 2014 8:31 PM
To: solr-user@lucene.apache.org
Subject: RE: suspect SOLR query from D029 (SOLR master)

We found a problem with the synonym list, and suspect there was some sort of
recursion causing the memory to be gobbled up until the JVM crashed.
Is this expected behavior from complex synonyms?
Or could this be due to the combination of complex synonyms and a bad query
format?


Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Friday, May 30, 2014 4:43 PM
To: solr-user@lucene.apache.org
Subject: RE: suspect SOLR query from D029 (SOLR master)

We've switched to CMS GC to see if there is any improvement.

Looking at this use case, G1GC might have been a better option, but we are
running JDK 1.6

http://blog.sematext.com/2013/06/24/g1-cms-java-garbage-collector/


Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Friday, May 30, 2014 11:52 AM
To: solr-user@lucene.apache.org
Subject: FW: suspect SOLR query from D029 (SOLR master)

We saw the file descriptors peak out and  full GCs running causing DOS on
our SOLR servers this morning.
* 

Cache response time

[Branham, Jeremy [HR]]

Is there a JMX metric for measuring the cache request time?

I can see the avg request times, but I'm assuming this includes the cache and 
non-cache values.

http://wiki.apache.org/solr/SolrPerformanceFactors






This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

RE: suspect SOLR query from D029 (SOLR master)

[Branham, Jeremy [HR]]

Evidently I didn't understand enough about the synonym filter.
I'm not sure anyone would be able to determine the impact based on the example 
queries below.

However I'm curious what the best practice is for synonyms.

We have 179 lines of synonyms each with 2 - 6 synonyms per line [all expanded].

The query that hurt us had 11 terms in the search string that matched a synonym 
line with 11 values.
How does this work internally?
Does the query parser create 11^11 search terms that would be parsed?
Should we be looking closer at stemming and tokenization rather than creating 
this list of synonyms?

Just looking for some clarity, or any best practice for use cases that might 
call for synonyms.


Thank you.

Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Monday, June 02, 2014 9:49 PM
To: solr-user@lucene.apache.org
Cc: Worley, Chris [HR]
Subject: RE: suspect SOLR query from D029 (SOLR master)

These are the typical queries we are using.
I'm curious if any of these parameters could be causing issues when using 
synonyms.



?shards=myserver1.com:8080/svc/solr/wdsc,myserver1.com:8080/svc/solr/kms&sort=score
 desc&q=(keyword:(this is a test) OR titleSearch:(this is a test) AND 
(doctype:("Device-Product")^1.1 OR (doctype:("LaunchPacks") OR 
doctype:("MenuMaps") OR doctype:("HowToSetupGuide") OR 
doctype:("ServiceAdvisory") OR doctype:("TroubleshootingGuide") OR 
doctype:("FAQ") OR 
doctype:("DeviceManual"&defType=edismax&fq=device_id:("Motorola Moto X 
\(XT1056\)")&fq=-doctype:Simulator AND 
-doctype:Scenario&bq=doctype:LaunchPacks^1.2 OR doctype:MenuMaps^1.2 OR 
doctype:HowToSetupGuide OR doctype:ServiceAdvisory^1.2 OR 
doctype:TroubleshootingGuide^1.4 OR doctype:FAQ^2.0 OR doctype:DeviceManual OR 
(doctype:Device-Product AND titleSearch:(Programming \-))^15 OR 
(doctype:Device-Product AND titleSearch:("Solution Matrix" \-))^5 OR 
(doctype:Device-Product AND titleSearch:("Fact Sheet" \-))^4 OR 
(doctype:Device-Product AND titleSearch:("User Guide" 
\-))^3&fl=id,title,device_id,url,doctype,titleSearch,score&mm=1&start=0&rows=10


?sort=score desc,hits30day 
desc&fq=profile:"YY"&fq=channel:ASPC_CustManaCM&q=this is a 
test&mm=0&start=0&rows=10


?fq=device_id:("Motorola Moto X \(XT1056\)")&q=(keyword:(this is a test) OR 
titleSearch:(this is a test))&start=0&rows=10


?sort=score desc&fq=doctype:ProductKnownIssue&fq=(technology:3G 1900/800, LTE 
both) &q=(text:Motorola Moto X (XT1056)^1000 OR text:[* TO *]) AND 
(titleSearch:(this is a test)^2 OR text:(this is a test)^1.5)&start=0&rows=10


?sort=score desc&q=((titleSearch:(this is a test)^2 OR text:(this is a test)) 
AND ((device_id:"Motorola Moto X \(XT1056\)" AND category:4247)^20 OR 
(category:3676 AND profile:"YY")))&fq=doctype:FlowDoc&fq=(-voice_network:[* TO 
*] AND *:*) OR voice_network:3G 1900/800, LTE&start=0&rows=10


?fq=doctype:Scenario&q=device_id:"Motorola Moto X (XT1056)" AND 
(titleSearch:(this is a test)^2 OR text:(this is a test)^1.5)&start=0&rows=10



Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Monday, June 02, 2014 8:31 PM
To: solr-user@lucene.apache.org
Subject: RE: suspect SOLR query from D029 (SOLR master)

We found a problem with the synonym list, and suspect there was some sort of 
recursion causing the memory to be gobbled up until the JVM crashed.
Is this expected behavior from complex synonyms?
Or could this be due to the combination of complex synonyms and a bad query 
format?


Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Friday, May 30, 2014 4:43 PM
To: solr-user@lucene.apache.org
Subject: RE: suspect SOLR query from D029 (SOLR master)

We've switched to CMS GC to see if there is any improvement.

Looking at this use case, G1GC might have been a better option, but we are 
running JDK 1.6

http://blog.sematext.com/2013/06/24/g1-cms-java-garbage-collector/


Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Friday, May 30, 2014 11:52 AM
To: solr-user@lucene.apache.org
Subject: FW: suspect SOLR query from D029 (SOLR master)

We saw the file descriptors peak out and  full GCs running causing DOS on our 
SOLR servers this morning.
* Does this stack trace give enough information for some ideas?

* solr-spec
4.5.1-SNAPSHOT
* solr-impl
4.5.1-SNAPSHOT ${svnversion} - kx101435 - 2013-11-04 17:39:36
* lucene-spec
4.5.1
* lucene-impl
4.5.1 1533280 - mark - 2013-10-17 21:40:03


We also saw some IO connection exceptions in the SOLR log -

2014-05-30 10:24:20

RE: suspect SOLR query from D029 (SOLR master)

[Branham, Jeremy [HR]]

These are the typical queries we are using.
I'm curious if any of these parameters could be causing issues when using 
synonyms.



?shards=myserver1.com:8080/svc/solr/wdsc,myserver1.com:8080/svc/solr/kms&sort=score
 desc&q=(keyword:(this is a test) OR titleSearch:(this is a test) AND 
(doctype:("Device-Product")^1.1 OR (doctype:("LaunchPacks") OR 
doctype:("MenuMaps") OR doctype:("HowToSetupGuide") OR 
doctype:("ServiceAdvisory") OR doctype:("TroubleshootingGuide") OR 
doctype:("FAQ") OR 
doctype:("DeviceManual"&defType=edismax&fq=device_id:("Motorola Moto X 
\(XT1056\)")&fq=-doctype:Simulator AND 
-doctype:Scenario&bq=doctype:LaunchPacks^1.2 OR doctype:MenuMaps^1.2 OR 
doctype:HowToSetupGuide OR doctype:ServiceAdvisory^1.2 OR 
doctype:TroubleshootingGuide^1.4 OR doctype:FAQ^2.0 OR doctype:DeviceManual OR 
(doctype:Device-Product AND titleSearch:(Programming \-))^15 OR 
(doctype:Device-Product AND titleSearch:("Solution Matrix" \-))^5 OR 
(doctype:Device-Product AND titleSearch:("Fact Sheet" \-))^4 OR 
(doctype:Device-Product AND titleSearch:("User Guide" 
\-))^3&fl=id,title,device_id,url,doctype,titleSearch,score&mm=1&start=0&rows=10


?sort=score desc,hits30day 
desc&fq=profile:"YY"&fq=channel:ASPC_CustManaCM&q=this is a 
test&mm=0&start=0&rows=10


?fq=device_id:("Motorola Moto X \(XT1056\)")&q=(keyword:(this is a test) OR 
titleSearch:(this is a test))&start=0&rows=10


?sort=score desc&fq=doctype:ProductKnownIssue&fq=(technology:3G 1900/800, LTE 
both) &q=(text:Motorola Moto X (XT1056)^1000 OR text:[* TO *]) AND 
(titleSearch:(this is a test)^2 OR text:(this is a test)^1.5)&start=0&rows=10


?sort=score desc&q=((titleSearch:(this is a test)^2 OR text:(this is a test)) 
AND ((device_id:"Motorola Moto X \(XT1056\)" AND category:4247)^20 OR 
(category:3676 AND profile:"YY")))&fq=doctype:FlowDoc&fq=(-voice_network:[* TO 
*] AND *:*) OR voice_network:3G 1900/800, LTE&start=0&rows=10


?fq=doctype:Scenario&q=device_id:"Motorola Moto X (XT1056)" AND 
(titleSearch:(this is a test)^2 OR text:(this is a test)^1.5)&start=0&rows=10



Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Monday, June 02, 2014 8:31 PM
To: solr-user@lucene.apache.org
Subject: RE: suspect SOLR query from D029 (SOLR master)

We found a problem with the synonym list, and suspect there was some sort of 
recursion causing the memory to be gobbled up until the JVM crashed.
Is this expected behavior from complex synonyms?
Or could this be due to the combination of complex synonyms and a bad query 
format?


Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Friday, May 30, 2014 4:43 PM
To: solr-user@lucene.apache.org
Subject: RE: suspect SOLR query from D029 (SOLR master)

We've switched to CMS GC to see if there is any improvement.

Looking at this use case, G1GC might have been a better option, but we are 
running JDK 1.6

http://blog.sematext.com/2013/06/24/g1-cms-java-garbage-collector/


Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Friday, May 30, 2014 11:52 AM
To: solr-user@lucene.apache.org
Subject: FW: suspect SOLR query from D029 (SOLR master)

We saw the file descriptors peak out and  full GCs running causing DOS on our 
SOLR servers this morning.
* Does this stack trace give enough information for some ideas?

* solr-spec
4.5.1-SNAPSHOT
* solr-impl
4.5.1-SNAPSHOT ${svnversion} - kx101435 - 2013-11-04 17:39:36
* lucene-spec
4.5.1
* lucene-impl
4.5.1 1533280 - mark - 2013-10-17 21:40:03


We also saw some IO connection exceptions in the SOLR log -

2014-05-30 10:24:20,334 ERROR - SolrDispatchFilter - 
null:org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,334 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http://myserver.com:8080/svc/solr/wdsp
2014-05-30 10:24:20,333 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,335 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,336 ERROR - SolrDis

RE: suspect SOLR query from D029 (SOLR master)

[Branham, Jeremy [HR]]

We found a problem with the synonym list, and suspect there was some sort of 
recursion causing the memory to be gobbled up until the JVM crashed.
Is this expected behavior from complex synonyms?
Or could this be due to the combination of complex synonyms and a bad query 
format?


Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Friday, May 30, 2014 4:43 PM
To: solr-user@lucene.apache.org
Subject: RE: suspect SOLR query from D029 (SOLR master)

We've switched to CMS GC to see if there is any improvement.

Looking at this use case, G1GC might have been a better option, but we are 
running JDK 1.6

http://blog.sematext.com/2013/06/24/g1-cms-java-garbage-collector/


Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Friday, May 30, 2014 11:52 AM
To: solr-user@lucene.apache.org
Subject: FW: suspect SOLR query from D029 (SOLR master)

We saw the file descriptors peak out and  full GCs running causing DOS on our 
SOLR servers this morning.
* Does this stack trace give enough information for some ideas?

* solr-spec
4.5.1-SNAPSHOT
* solr-impl
4.5.1-SNAPSHOT ${svnversion} - kx101435 - 2013-11-04 17:39:36
* lucene-spec
4.5.1
* lucene-impl
4.5.1 1533280 - mark - 2013-10-17 21:40:03


We also saw some IO connection exceptions in the SOLR log -

2014-05-30 10:24:20,334 ERROR - SolrDispatchFilter - 
null:org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,334 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http://myserver.com:8080/svc/solr/wdsp
2014-05-30 10:24:20,333 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,335 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,336 ERROR - SolrDispatchFilter - 
null:org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,335 ERROR - SolrDispatchFilter - 
null:org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsp



Jeremy D. Branham
Tel: **DOTNET

From: Worley, Chris [HR]
Sent: Friday, May 30, 2014 11:38 AM
To: Stoner, Susan [HR]; Branham, Jeremy [HR]
Cc: Barrett, Kevin B [HR]; Aldrich, Daniel J [HR]; Duncan, Horace W [HR]
Subject: suspect SOLR query from D029 (SOLR master)

Looks like a suspect SOLR query was executed at approx. 8:54am this morning on 
the SOLR master server (D029) that caused the java GC to go through the roof.  
See below:

2014-05-30 08:54:17,092 ERROR - SolrDispatchFilter - 
null:java.lang.OutOfMemoryError: GC overhead limit exceeded
at 
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadBlock(BlockTreeTermsReader.java:2377)
at 
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.seekExact(BlockTreeTermsReader.java:1698)
at org.apache.lucene.index.TermContext.build(TermContext.java:95)
at org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:166)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.FilteredQuery.createWeight(FilteredQuery.java:82)
at 
org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:690)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1501)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1367)
 

RE: suspect SOLR query from D029 (SOLR master)

[Branham, Jeremy [HR]]

We've switched to CMS GC to see if there is any improvement.

Looking at this use case, G1GC might have been a better option, but we are 
running JDK 1.6

http://blog.sematext.com/2013/06/24/g1-cms-java-garbage-collector/


Jeremy D. Branham
Tel: **DOTNET


-Original Message-
From: Branham, Jeremy [HR]
Sent: Friday, May 30, 2014 11:52 AM
To: solr-user@lucene.apache.org
Subject: FW: suspect SOLR query from D029 (SOLR master)

We saw the file descriptors peak out and  full GCs running causing DOS on our 
SOLR servers this morning.
* Does this stack trace give enough information for some ideas?

* solr-spec
4.5.1-SNAPSHOT
* solr-impl
4.5.1-SNAPSHOT ${svnversion} - kx101435 - 2013-11-04 17:39:36
* lucene-spec
4.5.1
* lucene-impl
4.5.1 1533280 - mark - 2013-10-17 21:40:03


We also saw some IO connection exceptions in the SOLR log -

2014-05-30 10:24:20,334 ERROR - SolrDispatchFilter - 
null:org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,334 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http://myserver.com:8080/svc/solr/wdsp
2014-05-30 10:24:20,333 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,335 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,336 ERROR - SolrDispatchFilter - 
null:org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,335 ERROR - SolrDispatchFilter - 
null:org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsp



Jeremy D. Branham
Tel: **DOTNET

From: Worley, Chris [HR]
Sent: Friday, May 30, 2014 11:38 AM
To: Stoner, Susan [HR]; Branham, Jeremy [HR]
Cc: Barrett, Kevin B [HR]; Aldrich, Daniel J [HR]; Duncan, Horace W [HR]
Subject: suspect SOLR query from D029 (SOLR master)

Looks like a suspect SOLR query was executed at approx. 8:54am this morning on 
the SOLR master server (D029) that caused the java GC to go through the roof.  
See below:

2014-05-30 08:54:17,092 ERROR - SolrDispatchFilter - 
null:java.lang.OutOfMemoryError: GC overhead limit exceeded
at 
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadBlock(BlockTreeTermsReader.java:2377)
at 
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.seekExact(BlockTreeTermsReader.java:1698)
at org.apache.lucene.index.TermContext.build(TermContext.java:95)
at org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:166)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.FilteredQuery.createWeight(FilteredQuery.java:82)
at 
org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:690)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1501)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1367)
at 
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:474)
at 
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:434)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at 
org.apache.solr.servlet.SolrDispatchFilter.ex

FW: suspect SOLR query from D029 (SOLR master)

[Branham, Jeremy [HR]]

We saw the file descriptors peak out and  full GCs running causing DOS on our 
SOLR servers this morning.
* Does this stack trace give enough information for some ideas?

* solr-spec
4.5.1-SNAPSHOT
* solr-impl
4.5.1-SNAPSHOT ${svnversion} - kx101435 - 2013-11-04 17:39:36
* lucene-spec
4.5.1
* lucene-impl
4.5.1 1533280 - mark - 2013-10-17 21:40:03


We also saw some IO connection exceptions in the SOLR log -

2014-05-30 10:24:20,334 ERROR - SolrDispatchFilter - 
null:org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,334 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http://myserver.com:8080/svc/solr/wdsp
2014-05-30 10:24:20,333 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,335 ERROR - SolrCore   - 
org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,336 ERROR - SolrDispatchFilter - 
null:org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsc
2014-05-30 10:24:20,335 ERROR - SolrDispatchFilter - 
null:org.apache.solr.common.SolrException: 
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: http:// myserver.com:8080/svc/solr/wdsp



Jeremy D. Branham
Tel: **DOTNET

From: Worley, Chris [HR]
Sent: Friday, May 30, 2014 11:38 AM
To: Stoner, Susan [HR]; Branham, Jeremy [HR]
Cc: Barrett, Kevin B [HR]; Aldrich, Daniel J [HR]; Duncan, Horace W [HR]
Subject: suspect SOLR query from D029 (SOLR master)

Looks like a suspect SOLR query was executed at approx. 8:54am this morning on 
the SOLR master server (D029) that caused the java GC to go through the roof.  
See below:

2014-05-30 08:54:17,092 ERROR - SolrDispatchFilter - 
null:java.lang.OutOfMemoryError: GC overhead limit exceeded
at 
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadBlock(BlockTreeTermsReader.java:2377)
at 
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.seekExact(BlockTreeTermsReader.java:1698)
at org.apache.lucene.index.TermContext.build(TermContext.java:95)
at org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:166)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:183)
at 
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:384)
at 
org.apache.lucene.search.FilteredQuery.createWeight(FilteredQuery.java:82)
at 
org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:690)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1501)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1367)
at 
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:474)
at 
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:434)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at

core swap duplicates core entries in solr.xml

[Branham, Jeremy [HR]]

When performing  a core swap in SOLR 4.5.1 with persistence on, the two core 
entries that were swapped are duplicated.

Solr.xml



  


















  



Performed swap -



  


























Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
Office: +1 (972) 405-2970 | Mobile: +1 (817) 791-1627
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham




This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

RE: SOLR Cloud node link is wrong in the admin panel

[Branham, Jeremy [HR]]

It seems the parameters in solr.xml are being ignored.



  

  




Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham


-Original Message-
From: Shawn Heisey [mailto:s...@elyograg.org]
Sent: Tuesday, October 22, 2013 3:14 PM
To: solr-user@lucene.apache.org
Subject: Re: SOLR Cloud node link is wrong in the admin panel

On 10/22/2013 2:01 PM, Branham, Jeremy [HR] wrote:
> I'm thinking I might have a configuration problem...
>
> The SOLR Cloud node link is wrong in the admin panel.
>
> I am running solr on port 8080 in JBOSS, but the SOLR cloud admin panel has 
> links to http://192.168.1.123:8983/solr for example.
> Also the context should be svc instead of solr.
>
> Is this a configuration problem, or are there some hardcoded values?

You're going to need to define the hostPort value in your solr.xml file.  In 
the example solr.xml, this is set to the following string:

${jetty.port:8983}

This means that it will use the java property "jetty.port" unless that's not 
defined, in which case it will use 8983.  Just remove this from hostPort and 
put 8080 in there.

http://wiki.apache.org/solr/SolrCloud#SolrCloud_Instance_Params

You might ask why Solr doesn't just figure out what port it is running on and 
store what it finds in the cloudstate.  The reason it doesn't do that is 
because it can't - a java webapp/servlet has no idea what port it's on until it 
actually receives a request, but it's not going to receive any requests until 
it's initialized, and by then it's too late to do anything useful with the 
information ... plus you need to send it a request.  This is one of the prime 
motivating factors behind the project's decision that Solr will no longer be a 
war in a future major version.

Thanks,
Shawn





This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

SOLR Cloud node link is wrong in the admin panel

[Branham, Jeremy [HR]]

I'm thinking I might have a configuration problem...

The SOLR Cloud node link is wrong in the admin panel.

I am running solr on port 8080 in JBOSS, but the SOLR cloud admin panel has 
links to http://192.168.1.123:8983/solr for example.
Also the context should be svc instead of solr.

Is this a configuration problem, or are there some hardcoded values?

Thanks,


Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham




This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

RE: External Zookeeper and JBOSS

[Branham, Jeremy [HR]]

[collections] was empty until I used the correct zkcli script from the solr 
distribution.

I uploaded the config -
java -classpath .:/production/v8p/deploy/svc.war/WEB-INF/lib/* 
org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost localhost:2181 -confdir 
/data/v8p/solr/root/conf -confname defaultconfig

Then ran the bootstrap -
java -classpath .:/production/v8p/deploy/svc.war/WEB-INF/lib/* 
org.apache.solr.cloud.ZkCLI -cmd bootstrap -zkhost 127.0.0.1:2181 -solrhome 
/data/v8p/solr

If I'm not mistaken, I don't need to link anything if the collection names are 
defined in the core element [solr.xml]
The cloud admin page shows each core now, but I'm curious how it know how many 
shards I want to use... I think I missed that somewhere.


Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham


-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Tuesday, October 22, 2013 3:57 AM
To: solr-user@lucene.apache.org
Subject: Re: External Zookeeper and JBOSS

What happens if you look in "collections"?

Best,
Erick


On Mon, Oct 21, 2013 at 9:55 PM, Shawn Heisey  wrote:

> On 10/21/2013 1:19 PM, Branham, Jeremy [HR] wrote:
>
>> Sorl.xml [simplified by removing additional cores]
>>
>>  > sharedLib="lib" zkHost="192.168.1.101:2181">
>>
>>  > instanceDir="/data/v8p/solr/**root/" name="wdsp"
>> dataDir="/data/v8p/solr/wdsp2/**data"/>
>>  > instanceDir="/data/v8p/solr/**root/" name="wdsp2"
>> dataDir="/data/v8p/solr/wdsp/**data"/>
>>
>> 
>>
>
> These cores that you have listed here do not look like
> SolrCloud-related cores, because they do not reference a collection or
> a shard.  Here's what I've got on a 4.2.1 box where all cores were
> automatically created by the CREATE action on the collections API:
>
>  instanceDir="eatatjoes_shard1_**replica2/" transient="false"
> name="eatatjoes_shard1_**replica2" config="solrconfig.xml"
> collection="eatatjoes"/>
>  instanceDir="test3_shard1_**replica1/" transient="false"
> name="test3_shard1_replica1" config="solrconfig.xml" collection="test3"/>
>  instanceDir="smb2_shard1_**replica1/" transient="false"
> name="smb2_shard1_replica1" config="solrconfig.xml"
> collection="smb2"/>
>
> On the commandline script -- the zkCli.sh script comes with zookeeper,
> but it is not aware of anything having to do with SolrCloud.  There is
> another script named zkcli.sh (note the lowercase C) that comes with
> the solr example (in example/cloud-scripts)- it's a very different
> script and will accept the options that you tried to give.
>
> I do wonder how much pain would be caused by renaming the Solr zkcli
> script so it's not so similar to the one that comes with Zookeeper.
>
> Thanks,
> Shawn
>
>



This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

RE: External Zookeeper and JBOSS

[Branham, Jeremy [HR]]

I've made progress...

Rather than using the zkCli.sh in the zookeep bin folder, I used the java libs 
fom SOLR and the config now shows up.




Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham


-Original Message-
From: Branham, Jeremy [HR]
Sent: Monday, October 21, 2013 2:20 PM
To: SOLR User distro (solr-user@lucene.apache.org)
Subject: External Zookeeper and JBOSS

When I use the Zookeeper CLI utility, I'm not sure if the configuration is 
uploading correctly.
How can I tell?

This is the command I am issuing -
./zkCli.sh -cmd upconfig -server 127.0.0.1:2181 -confdir 
/data/v8p/solr/root/conf -confname defaultconfig -solrhome /data/v8p/solr

Then checking with this -
[zk: localhost:2181(CONNECTED) 0] ls /
[aliases.json, live_nodes, overseer, overseer_elect, collections, zookeeper, 
clusterstate.json]


But I don't see any config node.

One thing to note - I have multiple cores but the configs are located in a 
common dir.
Maybe that is causing a problem.

Sorl.xml [simplified by removing additional cores]



  


  



Am I overlooking something obvious?

Thanks!



Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
http://JeremyBranham.Wordpress.com<http://jeremybranham.wordpress.com/>
http://www.linkedin.com/in/jeremybranham




This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.



This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

External Zookeeper and JBOSS

[Branham, Jeremy [HR]]

When I use the Zookeeper CLI utility, I'm not sure if the configuration is 
uploading correctly.
How can I tell?

This is the command I am issuing -
./zkCli.sh -cmd upconfig -server 127.0.0.1:2181 -confdir 
/data/v8p/solr/root/conf -confname defaultconfig -solrhome /data/v8p/solr

Then checking with this -
[zk: localhost:2181(CONNECTED) 0] ls /
[aliases.json, live_nodes, overseer, overseer_elect, collections, zookeeper, 
clusterstate.json]


But I don't see any config node.

One thing to note - I have multiple cores but the configs are located in a 
common dir.
Maybe that is causing a problem.

Sorl.xml [simplified by removing additional cores]



  


  



Am I overlooking something obvious?

Thanks!



Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham




This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

SOLR Cloud on JBOSS

[Branham, Jeremy [HR]]

Hello -

This wiki page is gone - https://wiki.apache.org/solr/SolrCloud%20using%20Jboss

I have been able to configure an external instance of Zookeeper, and an 
instance of SOLR in JBOSS..
But I am unsure how to point my SOLR instance to the ZK instance and upload the 
configuration.

All the examples I have found, show using script parameters to start SOLR 
rather than using a container like JBOSS.

Can someone point me in the right direction?

Thanks!


Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham




This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

RE: Best configuration for 2 servers

[Branham, Jeremy [HR]]

I may be interpreting this incorrectly, but shouldn't the cloud still serve 
requests if ZK crashes?

http://wiki.apache.org/solr/SolrCloud#Example_C:_Two_shard_cluster_with_shard_replicas_and_zookeeper_ensemble

" The problem with example B is that while there are enough Solr servers to 
survive any one of them crashing, there is only one zookeeper server that 
contains the state of the cluster. If that zookeeper server crashes, 
distributed queries will still work since the solr servers remember the state 
of the cluster last reported by zookeeper. The problem is that no new servers 
or clients will be able to discover the cluster state, and no changes to the 
cluster state will be possible."





Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
Office: +1 (972) 405-2970 | Mobile: +1 (817) 791-1627
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham


-Original Message-
From: Shawn Heisey [mailto:s...@elyograg.org]
Sent: Friday, September 13, 2013 2:40 PM
To: solr-user@lucene.apache.org
Subject: Re: Best configuration for 2 servers

On 9/13/2013 12:50 PM, Branham, Jeremy [HR] wrote:
> Does this sound appropriate then? [assuming no 3rd server]
>
> Server A:
> Zoo Keeper
> SOLR with 1 shard
>
> Server B:
> SOLR with ZK Host parameter set to Server A

Yes, that will work, but if the ZK on server A goes down, the entire cloud is 
down.

When you create a collection with replicationFactor=2, one replica will be on 
server A and one replica will be on server B.

If you want to break the index up into multiple shards, you can, you'll also 
need the maxShardsPerNode parameter when you create the collection, and all 
shards will have replicas on both machines.

A note about zookeeper and redundancy, and an explanation about why 3 hosts are 
required:  To form a quorum, zookeeper must have the votes of a majority of the 
hosts in the ensemble.  If there are only two hosts, it's not possible for 
there to be a majority unless both hosts are up, so two hosts is actually worse 
than one.  You need to either have one ZK node or at least three, preferably an 
odd number.

Thanks,
Shawn





This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

RE: Best configuration for 2 servers

[Branham, Jeremy [HR]]

Thanks Shawn -

Does this sound appropriate then? [assuming no 3rd server]

Server A:
Zoo Keeper
SOLR with 1 shard

Server B:
SOLR with ZK Host parameter set to Server A



Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
Office: +1 (972) 405-2970 | Mobile: +1 (817) 791-1627
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham


-Original Message-
From: Shawn Heisey [mailto:s...@elyograg.org]
Sent: Friday, September 13, 2013 11:48 AM
To: solr-user@lucene.apache.org
Subject: Re: Best configuration for 2 servers

On 9/13/2013 10:16 AM, Branham, Jeremy [HR] wrote:
> Currently, our SOLR 1.3 installation shares 4 applications servers with other 
> Java apps, leveraging master/slave replication.
>
> To get application isolation, we are moving from SOLR 1.3 to 4.3 and 
> acquiring 2 new production [vm] servers for the migration.
> For the new SOLR configuration, we are considering leveraging SOLR Cloud, but 
> there would be no shard redundancy with only 2 servers.
>
> Are there any good reasons to use a 2 shard cloud setup with no redundancy 
> versus a Master/Slave configuration on SOLR 4.3?

You should go to Solr 4.4, not 4.3.  Version 4.5 will be out soon, so unless 
you're going to go live before 2-3 weeks go by, you should probably plan on 
going with 4.5 instead.  Version 4.5 would be a
*really* good idea if you're going to use SolrCloud, as there are significant 
indexing improvements coming.

With SolrCloud, you can have shard redundancy with two servers.  All shards 
will exist on both servers.  What you won't have with only two servers is 
zookeeper redundancy, and zookeeper is *critical* for SolrCloud.  If you can 
add a third server with minimal CPU/RAM that's just for zookeeper, you can have 
that redundancy with no problem.  This is what I have done for my small cloud 
install.  It's the sort of thing that you could even just throw a desktop 
computer on the network to do, as long as you monitor it really well so you 
know if it ever goes down.

My large main Solr install is two completely independent sharded index copies 
*WITHOUT* SolrCloud.  It's been that way since Solr 3.x because master/slave 
replication is too inflexible.

Thanks,
Shawn





This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

Best configuration for 2 servers

[Branham, Jeremy [HR]]

Currently, our SOLR 1.3 installation shares 4 applications servers with other 
Java apps, leveraging master/slave replication.

To get application isolation, we are moving from SOLR 1.3 to 4.3 and acquiring 
2 new production [vm] servers for the migration.
For the new SOLR configuration, we are considering leveraging SOLR Cloud, but 
there would be no shard redundancy with only 2 servers.

Are there any good reasons to use a 2 shard cloud setup with no redundancy 
versus a Master/Slave configuration on SOLR 4.3?

Thanks!



Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham




This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

RE: Best configuration for 2 servers

[Branham, Jeremy [HR]]

Or another possible scenario -

SOLR Cloud with one logical shard and 2 servers would give me replication 
without the master/slave setup.
Is that correct?



Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham


-Original Message-
From: Branham, Jeremy [HR]
Sent: Friday, September 13, 2013 11:17 AM
To: SOLR User distro (solr-user@lucene.apache.org)
Subject: Best configuration for 2 servers

Currently, our SOLR 1.3 installation shares 4 applications servers with other 
Java apps, leveraging master/slave replication.

To get application isolation, we are moving from SOLR 1.3 to 4.3 and acquiring 
2 new production [vm] servers for the migration.
For the new SOLR configuration, we are considering leveraging SOLR Cloud, but 
there would be no shard redundancy with only 2 servers.

Are there any good reasons to use a 2 shard cloud setup with no redundancy 
versus a Master/Slave configuration on SOLR 4.3?

Thanks!



Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Tel: **DOTNET
http://JeremyBranham.Wordpress.com<http://jeremybranham.wordpress.com/>
http://www.linkedin.com/in/jeremybranham




This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.



This e-mail may contain Sprint proprietary information intended for the sole 
use of the recipient(s). Any use by others is prohibited. If you are not the 
intended recipient, please contact the sender and delete all copies of the 
message.

RE: SOLR reindexing

[Branham, Jeremy [HR]]

Thank Chris.
I considered this approach but wasn't sure about resource consumption.

We've run into a couple of issues where a full index rebuild/swap/replicate 
[overlapping] has left the slaves looking for an index that doesn't exist. This 
should resolve that issue.



Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Office: +1 (972) 405-2970 | Mobile: +1 (817) 791-1627
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham


-Original Message-
From: Chris Hostetter [mailto:hossman_luc...@fucit.org]
Sent: Friday, March 01, 2013 6:22 PM
To: solr-user@lucene.apache.org
Subject: Re: SOLR reindexing


: For full reindexes (DIH full-import), I use build cores, then swap them with
: the live cores.  I don't do this for performance reasons, I do it because I
: want to continue making incremental updates to the live cores while the
: rebuild is underway.  The rebuild takes four hours.

that's kind of a special case though -- the OP mentioned that the entire reason 
he does full rebuilds is because he has no way of inrementally tracking changes 
to his source data, so he's clearly not going to be making incremental updates 
to a "live" core.

in the simpler case of "rebuild the full index ever N hours, never to 
incremental updates" a simple master/slave setup is probably the easiest
-- with a single "snappull" command being triggered once you know the full 
index is build.

If you only have one machine to work with, then another simple appropach is 
just use a single solr core, and rebuild ontop of your existing data every N 
hours.  You can use a "timestamp" field to keep track of when documents were 
added and do a deleteByQuery on the timestamp field at the end of teh "rebuild" 
to remove any old documents (ie: things no longer in your source data)

as long as you don't commit until the end of your "rebuild" you don't need to 
worry about inconsistent data, and you should wind up using less resources then 
the core swapping approach.

-Hoss




This e-mail may contain Sprint Nextel proprietary information intended for the 
sole use of the recipient(s). Any use by others is prohibited. If you are not 
the intended recipient, please contact the sender and delete all copies of the 
message.

RE: SOLR reindexing

[Branham, Jeremy [HR]]

Thanks Shawn.



Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Office: +1 (972) 405-2970 | Mobile: +1 (817) 791-1627
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham


-Original Message-
From: Shawn Heisey [mailto:s...@elyograg.org]
Sent: Friday, March 01, 2013 2:31 PM
To: solr-user@lucene.apache.org
Subject: Re: SOLR reindexing

On 3/1/2013 12:01 PM, Branham, Jeremy [HR] wrote:
> I've read this...
> http://stackoverflow.com/questions/5154093/solr-requests-time-out-duri
> ng-index-update-perhaps-replication-a-possible-solut
>
> [Using SOLR 1.4]
>
> We are doing intraday full re-index because we aren't sure if the document 
> has been modified.
> Currently we are re-indexing to a second core then swapping to the original.
>
> Is this still the preferred method to avoid searcher delays?
>
> [We are planning an upgrade to SOLR 4 soon, if that makes any
> difference]

For full reindexes (DIH full-import), I use build cores, then swap them with 
the live cores.  I don't do this for performance reasons, I do it because I 
want to continue making incremental updates to the live cores while the rebuild 
is underway.  The rebuild takes four hours.

It's normal for searches to take longer while an index rebuild is happening.  
The extreme change that the stackoverflow user is seeing (queries going from 80 
milliseconds to over 1 milliseconds) suggests fundamental performance 
problems, possibly from an undersized server that doesn't have enough RAM for 
the OS to cache the index files.  With regular full rebuilds in another core, 
you'd actually want enough RAM to cache both the old index and the new one.

If you need to make visible updates to the existing index while building a new 
one, then your current practice of building in another core and swapping is the 
only reasonable solution.  I would probably stick to that model unless you're 
planning to move to SolrCloud, which works very differently and doesn't have 
collection swapping available.

Thanks,
Shawn





This e-mail may contain Sprint Nextel proprietary information intended for the 
sole use of the recipient(s). Any use by others is prohibited. If you are not 
the intended recipient, please contact the sender and delete all copies of the 
message.

SOLR reindexing

[Branham, Jeremy [HR]]

I've read this...
http://stackoverflow.com/questions/5154093/solr-requests-time-out-during-index-update-perhaps-replication-a-possible-solut

[Using SOLR 1.4]

We are doing intraday full re-index because we aren't sure if the document has 
been modified.
Currently we are re-indexing to a second core then swapping to the original.

Is this still the preferred method to avoid searcher delays?

[We are planning an upgrade to SOLR 4 soon, if that makes any difference]



Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Office: +1 (972) 405-2970 | Mobile: +1 (817) 791-1627
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham




This e-mail may contain Sprint Nextel proprietary information intended for the 
sole use of the recipient(s). Any use by others is prohibited. If you are not 
the intended recipient, please contact the sender and delete all copies of the 
message.