Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

[Jerome Yang]

What I'm doing is to simulate host crashed situation.

Consider this, a host is not connected to the cluster.

So, if a host crashed, I can not delete the down replicas by using
onlyIfDown='true'.
But in solr admin ui, it shows down for these replicas.
And whiteout "onlyIfDown", it still show a failure:
Delete replica failed: Attempted to remove replica :
demo.public.tbl/shard0/core_node4 with onlyIfDown='true', but state is
'active'.

Is this the right behavior? If a hosts gone, I can not delete replicas in
this host?

Regards,
Jerome

On Wed, Jul 20, 2016 at 1:58 AM, Justin Lee  wrote:

> Thanks for taking the time for the detailed response. I completely get what
> you are saying. Makes sense.
> On Tue, Jul 19, 2016 at 10:56 AM Erick Erickson 
> wrote:
>
> > Justin:
> >
> > Well, "kill -9" just makes it harder. The original question
> > was whether a replica being "active" was a bug, and it's
> > not when you kill -9; the Solr node has no chance to
> > tell Zookeeper it's going away. ZK does modify
> > the live_nodes by itself, thus there are checks as
> > necessary when a replica's state is referenced
> > whether the node is also in live_nodes. And an
> > overwhelming amount of the time this is OK, Solr
> > recovers just fine.
> >
> > As far as the write locks are concerned, those are
> > a Lucene level issue so if you kill Solr at just the
> > wrong time it's possible that that'll be left over. The
> > write locks are held for as short a period as possible
> > by Lucene, but occasionally they can linger if you kill
> > -9.
> >
> > When a replica comes up, if there is a write lock already, it
> > doesn't just take over; it fails to load instead.
> >
> > A kill -9 won't bring the cluster down by itself except
> > if there are several coincidences. Just don't make
> > it a habit. For instance, consider if you kill -9 on
> > two Solrs that happen to contain all of the replicas
> > for a shard1 for collection1. And you _happen_ to
> > kill them both at just the wrong time and they both
> > leave Lucene write locks for those replicas. Now
> > no replica will come up for shard1 and the collection
> > is unusable.
> >
> > So the shorter form is that using "kill -9" is a poor practice
> > that exposes you to some risk. The hard-core Solr
> > guys work extremely had to compensate for this kind
> > of thing, but kill -9 is a harsh, last-resort option and
> > shouldn't be part of your regular process. And you should
> > expect some "interesting" states when you do. And
> > you should use the bin/solr script to stop Solr
> > gracefully.
> >
> > Best,
> > Erick
> >
> >
> > On Tue, Jul 19, 2016 at 9:29 AM, Justin Lee 
> > wrote:
> > > Pardon me for hijacking the thread, but I'm curious about something you
> > > said, Erick.  I always thought that the point (in part) of going
> through
> > > the pain of using zookeeper and creating replicas was so that the
> system
> > > could seamlessly recover from catastrophic failures.  Wouldn't an OOM
> > > condition have a similar effect (or maybe java is better at cleanup on
> > that
> > > kind of error)?  The reason I ask is that I'm trying to set up a solr
> > > system that is highly available and I'm a little bit surprised that a
> > kill
> > > -9 on one process on one machine could put the entire system in a bad
> > > state.  Is it common to have to address problems like this with manual
> > > intervention in production systems?  Ideally, I'd hope to be able to
> set
> > up
> > > a system where a single node dying a horrible death would never require
> > > intervention.
> > >
> > > On Tue, Jul 19, 2016 at 8:54 AM Erick Erickson <
> erickerick...@gmail.com>
> > > wrote:
> > >
> > >> First of all, killing with -9 is A Very Bad Idea. You can
> > >> leave write lock files laying around. You can leave
> > >> the state in an "interesting" place. You haven't given
> > >> Solr a chance to tell Zookeeper that it's going away.
> > >> (which would set the state to "down"). In short
> > >> when you do this you have to deal with the consequences
> > >> yourself, one of which is this mismatch between
> > >> cluster state and live_nodes.
> > >>
> > >> Now, that rant done the bin/solr script tries to stop Solr
> > >> gracefully but issues a kill if solr doesn't stop nicely. Personally
> > >> I think that timeout should be longer, but that's another story.
> > >>
> > >> The onlyIfDown='true' option is there specifically as a
> > >> safety valve. It was provided for those who want to guard against
> > >> typos and the like, so just don't specify it and you should be fine.
> > >>
> > >> Best,
> > >> Erick
> > >>
> > >> On Mon, Jul 18, 2016 at 11:51 PM, Jerome Yang 
> > wrote:
> > >> > Hi all,
> > >> >
> > >> > Here's the situation.
> > >> > I'm using solr5.3 in cloud mode.
> > >> >
> > >> > I have 4 nodes.
> > >> >
> > >> > After use "kill -9 pid-solr-node" to kill 2 nodes.
> > >> > These replicas in 

Re: Solr Cloud - how to implement local indexing without SSL and distributed search with SSL

[Shawn Heisey]

On 7/17/2016 8:13 AM, Sarit Weber wrote:
> We noticed that indexing is much faster without SSL, but we can not
> remove it from distributed search. 

Solr doesn't handle the networking.  That's Jetty.  Jetty sets up one
listening port, and that port either uses SSL or it doesn't.  All
requests for Solr are handled by that network setup, which is out of
Solr's control.

To use SSL for one part of the app but not for another part would
require that somebody split Solr into *two* applications, and have jetty
handle each application with a different TCP port.  Then the two
applications would need a way to talk to each other.  I can almost
guarantee that the Solr developers are not going to implement that.

You might want to remove SSL from Solr and have your query clients
access it via a proxy/loadbalancer that handles the SSL.  Because you're
looking for this kind of solution, I imagine that the clients that will
be indexing are very different clients than the ones that are doing the
queries, and that maybe the clients doing the queries are in a network
that you do not trust.

One final word -- Solr should not be accessible from any network
location that you cannot completely trust, especially the Internet.

Thanks,
Shawn

Re: using lucene parser syntax with eDisMax

[Chris Hostetter]

: Yes on both counts. Although it takes a bit of practice, if you add
: =query to the query you'll see a section of the
: response showing you exactly what the resulting query is after
: all the rules are applied.

In addition, something else about edismax that you might find useful (but 
isn't immediatley obvious) ...

: > The documentation at 
: > 
https://cwiki.apache.org/confluence/display/solr/The+Extended+DisMax+Query+Parser
 
: > says that eDisMax "supports the full Lucene query parser syntax".
: > Does this mean that a query string "color:brown^2 and mazda" is legal 
: > with eDisMax too?  Notice that I am specifying the color field in the 
: > query (lucene parser syntax). If the answer is yes, does this mean that 
: > "brown" is only filtered against the color field and mazda will be 
: > filtered against both the color field and the brand field?

...with the "uf" param, and per field "qf" overrides, you can define what 
it means when a query string contains a fieldname like "color" -- even if 
your index doesn't have a field named "color".

with something like this...

   defType=edismax
   uf=color title brand
   qf=main_color trim_color brand^5 title^10 text
   f.color.qf=main_color^3 trim_color^2
   q=color:brown mazda

..."brown" would only be searched against the main_color and 
trim_color fields (with the specified boosts) while "mazda" would be 
searched against all the fields listed in the "qf".  

As long as you use the "uf" param, it limits the "field names" (or field 
"aliases" if you use the f.foo.qf syntax) a user can specify in the "q" 
param -- so if you had all the params mentioned above, but the query was...

  q=color:brown security_level:private

...then even if you have a "security_level" field in your index, that part 
of the query string will be treated as a plain old string value and not a 
field name -- so all your qf fields will be searched for the string 
"security_level:private"



-Hoss
http://www.lucidworks.com/

Re: Solr dual core performance

[Erick Erickson]

I strongly suspect you're not getting "real" searches, but
are hitting your query result cache or perhaps some other
cache. 1.24ms response times are quite unusual.

So check the Solr queryResultCache hit ratio, whether any
fronting HTTP caching is being hit and the like would be
my first step.

Allocating 50G of 60G RAM to Solr is an anti-pattern, see:
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html

Your CPU usage is indicating that Solr isn't doing much at all, also
indicating that you aren't doing "real" searches unless your load
test is sending the queries serially.

And I'm not quite sure what you mean when you say "Dual core".
If you're talking about having two _Solr_ cores on the same machine,
they likely share the disk I/O. Since each response needs to read
data from disk (for stored fields), disk I/O would be a likely place to look.

Best,
Erick

On Tue, Jul 19, 2016 at 12:10 PM, Shweta Udapudi  wrote:
> Most important information
>
> solr-spec 5.4.1
> solr-impl 5.4.1 1725212 - jpountz - 2016-01-18 11:51:45
> lucene-spec 5.4.1
> lucene-impl 5.4.1 1725212 - jpountz - 2016-01-18 11:44:59
>
> java version "1.7.0_79"
> OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-0ubuntu1.12.04.1)
> OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)
>
>
> -Original Message-
> From: Shweta Udapudi
> Sent: Tuesday, July 19, 2016 12:08 PM
> To: 'solr-user@lucene.apache.org' 
> Subject: Solr dual core performance
>
> Hi All,
>
> I have Solr Server on hardware 60GB RAM split 50GB Solr RAM and the OS. The 
> search index size is 120GB and built offline. There are no updates to this 
> index. I have 2 cores setup, they are completely identical. Except they are 
> on 2 different disk drives.
>
> The test run with the same 3 million queries/core.
> The medianRequestTime went from 1.24ms to 25ms. Performance is 20X worse.
>
> Examined
> - RAM usage and GC - very few full GC and nothing stands out as unusual
> - CPU usage is about 10%
> - Disk IO (currently  collecting data)
>
> What other components should I be looking at? Is there any shared resource 
> between 2 disks?
> Solr reported statistics below.
>
> Thanks
> Shweta
>
> Single Core
> CORE1
> requests:   3077953
> errors: 1580
> timeouts:   0
> totalTime:  115427000
> avgRequestsPerSecond:   840.071468
> 5minRateReqsPerSecond:  371.1174956
> 15minRateReqsPerSecond: 871.0010528
> avgTimePerRequest:  37.50122241
> medianRequestTime:  1.245865
> 75thPcRequestTime:  1.9472615
> 95thPcRequestTime:  4.74148915
> 99thPcRequestTime:  7.27607746
> 999thPcRequestTime: 465.8678296
>
>
> Dual Core
> CORE1
> requests:   3077953
> errors: 1580
> timeouts:   0
> totalTime:  485785273.2
> avgRequestsPerSecond:   52.17154443
> 5minRateReqsPerSecond:  6.17E-53
> 15minRateReqsPerSecond: 1.21E-16
> avgTimePerRequest:  157.827385
> medianRequestTime:  25.30935
> 75thPcRequestTime:  64.17734025
> 95thPcRequestTime:  209.5386719
> 99thPcRequestTime:  382.1332734
> 999thPcRequestTime: 960.208626
>
> CORE2
> requests:   3077953
> errors: 1580
> timeouts:   0
> totalTime:  485785273.2
> avgRequestsPerSecond:   52.60462395
> 5minRateReqsPerSecond:  3.11E-52
> 15minRateReqsPerSecond: 2.07E-16
> avgTimePerRequest:  157.827385
> medianRequestTime:  25.30935
> 75thPcRequestTime:  64.17734025
> 95thPcRequestTime:  209.5386719
> 99thPcRequestTime:  382.1332734
> 999thPcRequestTime: 960.208626
>
>
>
>
>
>

RE: Solr dual core performance

[Shweta Udapudi]

Most important information 

solr-spec 5.4.1
solr-impl 5.4.1 1725212 - jpountz - 2016-01-18 11:51:45
lucene-spec 5.4.1
lucene-impl 5.4.1 1725212 - jpountz - 2016-01-18 11:44:59

java version "1.7.0_79"
OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-0ubuntu1.12.04.1)
OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)


-Original Message-
From: Shweta Udapudi 
Sent: Tuesday, July 19, 2016 12:08 PM
To: 'solr-user@lucene.apache.org' 
Subject: Solr dual core performance 

Hi All,

I have Solr Server on hardware 60GB RAM split 50GB Solr RAM and the OS. The 
search index size is 120GB and built offline. There are no updates to this 
index. I have 2 cores setup, they are completely identical. Except they are on 
2 different disk drives.

The test run with the same 3 million queries/core. 
The medianRequestTime went from 1.24ms to 25ms. Performance is 20X worse. 

Examined 
- RAM usage and GC - very few full GC and nothing stands out as unusual
- CPU usage is about 10% 
- Disk IO (currently  collecting data)

What other components should I be looking at? Is there any shared resource 
between 2 disks? 
Solr reported statistics below.

Thanks
Shweta

Single Core 
CORE1
requests:   3077953
errors: 1580
timeouts:   0
totalTime:  115427000
avgRequestsPerSecond:   840.071468
5minRateReqsPerSecond:  371.1174956
15minRateReqsPerSecond: 871.0010528
avgTimePerRequest:  37.50122241
medianRequestTime:  1.245865
75thPcRequestTime:  1.9472615
95thPcRequestTime:  4.74148915
99thPcRequestTime:  7.27607746
999thPcRequestTime: 465.8678296


Dual Core 
CORE1
requests:   3077953
errors: 1580
timeouts:   0
totalTime:  485785273.2
avgRequestsPerSecond:   52.17154443
5minRateReqsPerSecond:  6.17E-53
15minRateReqsPerSecond: 1.21E-16
avgTimePerRequest:  157.827385
medianRequestTime:  25.30935
75thPcRequestTime:  64.17734025
95thPcRequestTime:  209.5386719
99thPcRequestTime:  382.1332734
999thPcRequestTime: 960.208626

CORE2
requests:   3077953
errors: 1580
timeouts:   0
totalTime:  485785273.2
avgRequestsPerSecond:   52.60462395
5minRateReqsPerSecond:  3.11E-52
15minRateReqsPerSecond: 2.07E-16
avgTimePerRequest:  157.827385
medianRequestTime:  25.30935
75thPcRequestTime:  64.17734025
95thPcRequestTime:  209.5386719
99thPcRequestTime:  382.1332734
999thPcRequestTime: 960.208626

Solr dual core performance

[Shweta Udapudi]

Hi All,

I have Solr Server on hardware 60GB RAM split 50GB Solr RAM and the OS. The 
search index size is 120GB and built offline. There are no updates to this 
index. I have 2 cores setup, they are completely identical. Except they are on 
2 different disk drives.

The test run with the same 3 million queries/core. 
The medianRequestTime went from 1.24ms to 25ms. Performance is 20X worse. 

Examined 
- RAM usage and GC - very few full GC and nothing stands out as unusual
- CPU usage is about 10% 
- Disk IO (currently  collecting data)

What other components should I be looking at? Is there any shared resource 
between 2 disks? 
Solr reported statistics below.

Thanks
Shweta

Single Core 
CORE1
requests:   3077953
errors: 1580
timeouts:   0
totalTime:  115427000
avgRequestsPerSecond:   840.071468
5minRateReqsPerSecond:  371.1174956
15minRateReqsPerSecond: 871.0010528
avgTimePerRequest:  37.50122241
medianRequestTime:  1.245865
75thPcRequestTime:  1.9472615
95thPcRequestTime:  4.74148915
99thPcRequestTime:  7.27607746
999thPcRequestTime: 465.8678296


Dual Core 
CORE1
requests:   3077953
errors: 1580
timeouts:   0
totalTime:  485785273.2
avgRequestsPerSecond:   52.17154443
5minRateReqsPerSecond:  6.17E-53
15minRateReqsPerSecond: 1.21E-16
avgTimePerRequest:  157.827385
medianRequestTime:  25.30935
75thPcRequestTime:  64.17734025
95thPcRequestTime:  209.5386719
99thPcRequestTime:  382.1332734
999thPcRequestTime: 960.208626

CORE2
requests:   3077953
errors: 1580
timeouts:   0
totalTime:  485785273.2
avgRequestsPerSecond:   52.60462395
5minRateReqsPerSecond:  3.11E-52
15minRateReqsPerSecond: 2.07E-16
avgTimePerRequest:  157.827385
medianRequestTime:  25.30935
75thPcRequestTime:  64.17734025
95thPcRequestTime:  209.5386719
99thPcRequestTime:  382.1332734
999thPcRequestTime: 960.208626

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

[Justin Lee]

Thanks for taking the time for the detailed response. I completely get what
you are saying. Makes sense.
On Tue, Jul 19, 2016 at 10:56 AM Erick Erickson 
wrote:

> Justin:
>
> Well, "kill -9" just makes it harder. The original question
> was whether a replica being "active" was a bug, and it's
> not when you kill -9; the Solr node has no chance to
> tell Zookeeper it's going away. ZK does modify
> the live_nodes by itself, thus there are checks as
> necessary when a replica's state is referenced
> whether the node is also in live_nodes. And an
> overwhelming amount of the time this is OK, Solr
> recovers just fine.
>
> As far as the write locks are concerned, those are
> a Lucene level issue so if you kill Solr at just the
> wrong time it's possible that that'll be left over. The
> write locks are held for as short a period as possible
> by Lucene, but occasionally they can linger if you kill
> -9.
>
> When a replica comes up, if there is a write lock already, it
> doesn't just take over; it fails to load instead.
>
> A kill -9 won't bring the cluster down by itself except
> if there are several coincidences. Just don't make
> it a habit. For instance, consider if you kill -9 on
> two Solrs that happen to contain all of the replicas
> for a shard1 for collection1. And you _happen_ to
> kill them both at just the wrong time and they both
> leave Lucene write locks for those replicas. Now
> no replica will come up for shard1 and the collection
> is unusable.
>
> So the shorter form is that using "kill -9" is a poor practice
> that exposes you to some risk. The hard-core Solr
> guys work extremely had to compensate for this kind
> of thing, but kill -9 is a harsh, last-resort option and
> shouldn't be part of your regular process. And you should
> expect some "interesting" states when you do. And
> you should use the bin/solr script to stop Solr
> gracefully.
>
> Best,
> Erick
>
>
> On Tue, Jul 19, 2016 at 9:29 AM, Justin Lee 
> wrote:
> > Pardon me for hijacking the thread, but I'm curious about something you
> > said, Erick.  I always thought that the point (in part) of going through
> > the pain of using zookeeper and creating replicas was so that the system
> > could seamlessly recover from catastrophic failures.  Wouldn't an OOM
> > condition have a similar effect (or maybe java is better at cleanup on
> that
> > kind of error)?  The reason I ask is that I'm trying to set up a solr
> > system that is highly available and I'm a little bit surprised that a
> kill
> > -9 on one process on one machine could put the entire system in a bad
> > state.  Is it common to have to address problems like this with manual
> > intervention in production systems?  Ideally, I'd hope to be able to set
> up
> > a system where a single node dying a horrible death would never require
> > intervention.
> >
> > On Tue, Jul 19, 2016 at 8:54 AM Erick Erickson 
> > wrote:
> >
> >> First of all, killing with -9 is A Very Bad Idea. You can
> >> leave write lock files laying around. You can leave
> >> the state in an "interesting" place. You haven't given
> >> Solr a chance to tell Zookeeper that it's going away.
> >> (which would set the state to "down"). In short
> >> when you do this you have to deal with the consequences
> >> yourself, one of which is this mismatch between
> >> cluster state and live_nodes.
> >>
> >> Now, that rant done the bin/solr script tries to stop Solr
> >> gracefully but issues a kill if solr doesn't stop nicely. Personally
> >> I think that timeout should be longer, but that's another story.
> >>
> >> The onlyIfDown='true' option is there specifically as a
> >> safety valve. It was provided for those who want to guard against
> >> typos and the like, so just don't specify it and you should be fine.
> >>
> >> Best,
> >> Erick
> >>
> >> On Mon, Jul 18, 2016 at 11:51 PM, Jerome Yang 
> wrote:
> >> > Hi all,
> >> >
> >> > Here's the situation.
> >> > I'm using solr5.3 in cloud mode.
> >> >
> >> > I have 4 nodes.
> >> >
> >> > After use "kill -9 pid-solr-node" to kill 2 nodes.
> >> > These replicas in the two nodes still are "ACTIVE" in zookeeper's
> >> > state.json.
> >> >
> >> > The problem is, when I try to delete these down replicas with
> >> > parameter onlyIfDown='true'.
> >> > It says,
> >> > "Delete replica failed: Attempted to remove replica :
> >> > demo.public.tbl/shard0/core_node4 with onlyIfDown='true', but state is
> >> > 'active'."
> >> >
> >> > From this link:
> >> > <
> >>
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
> >> >
> >> > <
> >>
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
> >> >
> >> > <
> >>
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
> >> >
> >> > <
> >>
> 

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

[Erick Erickson]

Justin:

Well, "kill -9" just makes it harder. The original question
was whether a replica being "active" was a bug, and it's
not when you kill -9; the Solr node has no chance to
tell Zookeeper it's going away. ZK does modify
the live_nodes by itself, thus there are checks as
necessary when a replica's state is referenced
whether the node is also in live_nodes. And an
overwhelming amount of the time this is OK, Solr
recovers just fine.

As far as the write locks are concerned, those are
a Lucene level issue so if you kill Solr at just the
wrong time it's possible that that'll be left over. The
write locks are held for as short a period as possible
by Lucene, but occasionally they can linger if you kill
-9.

When a replica comes up, if there is a write lock already, it
doesn't just take over; it fails to load instead.

A kill -9 won't bring the cluster down by itself except
if there are several coincidences. Just don't make
it a habit. For instance, consider if you kill -9 on
two Solrs that happen to contain all of the replicas
for a shard1 for collection1. And you _happen_ to
kill them both at just the wrong time and they both
leave Lucene write locks for those replicas. Now
no replica will come up for shard1 and the collection
is unusable.

So the shorter form is that using "kill -9" is a poor practice
that exposes you to some risk. The hard-core Solr
guys work extremely had to compensate for this kind
of thing, but kill -9 is a harsh, last-resort option and
shouldn't be part of your regular process. And you should
expect some "interesting" states when you do. And
you should use the bin/solr script to stop Solr
gracefully.

Best,
Erick


On Tue, Jul 19, 2016 at 9:29 AM, Justin Lee  wrote:
> Pardon me for hijacking the thread, but I'm curious about something you
> said, Erick.  I always thought that the point (in part) of going through
> the pain of using zookeeper and creating replicas was so that the system
> could seamlessly recover from catastrophic failures.  Wouldn't an OOM
> condition have a similar effect (or maybe java is better at cleanup on that
> kind of error)?  The reason I ask is that I'm trying to set up a solr
> system that is highly available and I'm a little bit surprised that a kill
> -9 on one process on one machine could put the entire system in a bad
> state.  Is it common to have to address problems like this with manual
> intervention in production systems?  Ideally, I'd hope to be able to set up
> a system where a single node dying a horrible death would never require
> intervention.
>
> On Tue, Jul 19, 2016 at 8:54 AM Erick Erickson 
> wrote:
>
>> First of all, killing with -9 is A Very Bad Idea. You can
>> leave write lock files laying around. You can leave
>> the state in an "interesting" place. You haven't given
>> Solr a chance to tell Zookeeper that it's going away.
>> (which would set the state to "down"). In short
>> when you do this you have to deal with the consequences
>> yourself, one of which is this mismatch between
>> cluster state and live_nodes.
>>
>> Now, that rant done the bin/solr script tries to stop Solr
>> gracefully but issues a kill if solr doesn't stop nicely. Personally
>> I think that timeout should be longer, but that's another story.
>>
>> The onlyIfDown='true' option is there specifically as a
>> safety valve. It was provided for those who want to guard against
>> typos and the like, so just don't specify it and you should be fine.
>>
>> Best,
>> Erick
>>
>> On Mon, Jul 18, 2016 at 11:51 PM, Jerome Yang  wrote:
>> > Hi all,
>> >
>> > Here's the situation.
>> > I'm using solr5.3 in cloud mode.
>> >
>> > I have 4 nodes.
>> >
>> > After use "kill -9 pid-solr-node" to kill 2 nodes.
>> > These replicas in the two nodes still are "ACTIVE" in zookeeper's
>> > state.json.
>> >
>> > The problem is, when I try to delete these down replicas with
>> > parameter onlyIfDown='true'.
>> > It says,
>> > "Delete replica failed: Attempted to remove replica :
>> > demo.public.tbl/shard0/core_node4 with onlyIfDown='true', but state is
>> > 'active'."
>> >
>> > From this link:
>> > <
>> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
>> >
>> > <
>> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
>> >
>> > <
>> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
>> >
>> > <
>> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
>> >
>> >
>> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
>> >
>> > It says:
>> > *NOTE*: when the node the replica is hosted on crashes, the replica's
>> state
>> > may remain ACTIVE in ZK. To determine if the replica is truly active, you
>> > must also verify that its node
>> > <
>> 

Re: Find part of long query in shorter fields

[CA]

Just for the records:

After realizing that with „defType=dismax“ I really do get the expected output 
I’ve found out what I need to change in my edismax configuration:

false

Then this will work:
> q=Braun Series 9 9095CC Men's Electric Shaver Wet/Dry with Clean and Renew 
> Charger
> // edismax with qf/pf : „name“ and „brand“ field
> 
Not returned anymore:
> name: "Braun Series Clean CCR2 Cleansing Dock Cartridges Lemonfresh 
> Formula Cartrige (Compatible with Series 7,5,3) 2 pc“
> brand: Braun
> 
Best hit:
> name: "Braun 9095cc Series 9 Electric Shaver“
> brand: Braun


Actually, as I’d like to disable operators in the query altogether (if 
possible), I’m wondering whether I should not be using the old dismax in the 
first place.

Cheers,
Chantal

Re: Node not recovering, leader elections not occuring

[Jeff Wartes]

It sounds like the node-local version of the ZK clusterstate has diverged from 
the ZK cluster state. You should check the contents of zookeeper and verify the 
state there looks sane. I’ve had issues (v5.4) on a few occasions where leader 
election got screwed up to the point where I had to delete data from ZK 
manually. (Usually after a ZK issue.)

Particularly, take a look in 
collections//leader_elect/shard/election

Non-sane states would be the same core_nodeX listed more than once, or fewer 
entries than (up) replicas. If you’re having trouble getting a sane election, 
you can try deleting the lowest-numbered entries (as well as any lower-numbered 
duplicates) and trying forceelection again. Possibly followed by restarting the 
node with that lowest-numbered entry.

Also make sure that this exists and has the expected replica
collections//leader/shard

collections//leader_initiated_recovery 
can be informative too, this represents replicas that the *leader* thinks are 
out of sync, usually due to a failed update request.


On 7/19/16, 9:20 AM, "Tom Evans"  wrote:

On the nodes that have the replica in a recovering state we now see:

19-07-2016 16:18:28 ERROR RecoveryStrategy:159 - Error while trying to
recover. core=lookups_shard1_replica8:org.apache.solr.common.SolrException:
No registered leader was found after waiting for 4000ms , collection:
lookups slice: shard1
at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:607)
at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:593)
at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:308)
at 
org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:224)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

19-07-2016 16:18:28 INFO  RecoveryStrategy:444 - Replay not started,
or was not successful... still buffering updates.
19-07-2016 16:18:28 ERROR RecoveryStrategy:481 - Recovery failed -
trying again... (164)
19-07-2016 16:18:28 INFO  RecoveryStrategy:503 - Wait [12.0] seconds
before trying to recover again (attempt=165)


This is with the "leader that is not the leader" shut down.

Issuing a FORCELEADER via collections API doesn't in fact force a
leader election to occur.

Is there any other way to prompt Solr to have an election?

Cheers

Tom

On Tue, Jul 19, 2016 at 5:10 PM, Tom Evans  wrote:
> There are 11 collections, each only has one shard, and each node has
> 10 replicas (9 collections are on every node, 2 are just on one node).
> We're not seeing any OOM errors on restart.
>
> I think we're being patient waiting for the leader election to occur.
> We stopped the troublesome "leader that is not the leader" server
> about 15-20 minutes ago, but we still have not had a leader election.
>
> Cheers
>
> Tom
>
> On Tue, Jul 19, 2016 at 4:30 PM, Erick Erickson  
wrote:
>> How many replicas per Solr JVM? And do you
>> see any OOM errors when you bounce a server?
>> And how patient are you being, because it can
>> take 3 minutes for a leaderless shard to decide
>> it needs to elect a leader.
>>
>> See SOLR-7280 and SOLR-7191 for the case
>> where lots of replicas are in the same JVM,
>> the tell-tale symptom is errors in the log as you
>> bring Solr up saying something like
>> "OutOfMemory error unable to create native thread"
>>
>> SOLR-7280 has patches for 6x and 7x, with a 5x one
>> being added momentarily.
>>
>> Best,
>> Erick
>>
>> On Tue, Jul 19, 2016 at 7:41 AM, Tom Evans  
wrote:
>>> Hi all - problem with a SolrCloud 5.5.0, we have a node that has most
>>> of the collections on it marked as "Recovering" or "Recovery Failed".
>>> It attempts to recover from the leader, but the leader responds with:
>>>
>>> Error while trying to recover.
>>> core=iris_shard1_replica1:java.util.concurrent.ExecutionException:
>>> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
>>> Error from server at http://172.31.1.171:3/solr: We are not the
>>> leader
>>> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>>> at 

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

[Justin Lee]

Pardon me for hijacking the thread, but I'm curious about something you
said, Erick.  I always thought that the point (in part) of going through
the pain of using zookeeper and creating replicas was so that the system
could seamlessly recover from catastrophic failures.  Wouldn't an OOM
condition have a similar effect (or maybe java is better at cleanup on that
kind of error)?  The reason I ask is that I'm trying to set up a solr
system that is highly available and I'm a little bit surprised that a kill
-9 on one process on one machine could put the entire system in a bad
state.  Is it common to have to address problems like this with manual
intervention in production systems?  Ideally, I'd hope to be able to set up
a system where a single node dying a horrible death would never require
intervention.

On Tue, Jul 19, 2016 at 8:54 AM Erick Erickson 
wrote:

> First of all, killing with -9 is A Very Bad Idea. You can
> leave write lock files laying around. You can leave
> the state in an "interesting" place. You haven't given
> Solr a chance to tell Zookeeper that it's going away.
> (which would set the state to "down"). In short
> when you do this you have to deal with the consequences
> yourself, one of which is this mismatch between
> cluster state and live_nodes.
>
> Now, that rant done the bin/solr script tries to stop Solr
> gracefully but issues a kill if solr doesn't stop nicely. Personally
> I think that timeout should be longer, but that's another story.
>
> The onlyIfDown='true' option is there specifically as a
> safety valve. It was provided for those who want to guard against
> typos and the like, so just don't specify it and you should be fine.
>
> Best,
> Erick
>
> On Mon, Jul 18, 2016 at 11:51 PM, Jerome Yang  wrote:
> > Hi all,
> >
> > Here's the situation.
> > I'm using solr5.3 in cloud mode.
> >
> > I have 4 nodes.
> >
> > After use "kill -9 pid-solr-node" to kill 2 nodes.
> > These replicas in the two nodes still are "ACTIVE" in zookeeper's
> > state.json.
> >
> > The problem is, when I try to delete these down replicas with
> > parameter onlyIfDown='true'.
> > It says,
> > "Delete replica failed: Attempted to remove replica :
> > demo.public.tbl/shard0/core_node4 with onlyIfDown='true', but state is
> > 'active'."
> >
> > From this link:
> > <
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
> >
> > <
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
> >
> > <
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
> >
> > <
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
> >
> >
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
> >
> > It says:
> > *NOTE*: when the node the replica is hosted on crashes, the replica's
> state
> > may remain ACTIVE in ZK. To determine if the replica is truly active, you
> > must also verify that its node
> > <
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.html#getNodeName--
> >
> > is
> > under /live_nodes in ZK (or use ClusterState.liveNodesContain(String)
> > <
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/ClusterState.html#liveNodesContain-java.lang.String-
> >
> > ).
> >
> > So, is this a bug?
> >
> > Regards,
> > Jerome
>

Re: Node not recovering, leader elections not occuring

[Tom Evans]

On the nodes that have the replica in a recovering state we now see:

19-07-2016 16:18:28 ERROR RecoveryStrategy:159 - Error while trying to
recover. core=lookups_shard1_replica8:org.apache.solr.common.SolrException:
No registered leader was found after waiting for 4000ms , collection:
lookups slice: shard1
at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:607)
at 
org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:593)
at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:308)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:224)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

19-07-2016 16:18:28 INFO  RecoveryStrategy:444 - Replay not started,
or was not successful... still buffering updates.
19-07-2016 16:18:28 ERROR RecoveryStrategy:481 - Recovery failed -
trying again... (164)
19-07-2016 16:18:28 INFO  RecoveryStrategy:503 - Wait [12.0] seconds
before trying to recover again (attempt=165)


This is with the "leader that is not the leader" shut down.

Issuing a FORCELEADER via collections API doesn't in fact force a
leader election to occur.

Is there any other way to prompt Solr to have an election?

Cheers

Tom

On Tue, Jul 19, 2016 at 5:10 PM, Tom Evans  wrote:
> There are 11 collections, each only has one shard, and each node has
> 10 replicas (9 collections are on every node, 2 are just on one node).
> We're not seeing any OOM errors on restart.
>
> I think we're being patient waiting for the leader election to occur.
> We stopped the troublesome "leader that is not the leader" server
> about 15-20 minutes ago, but we still have not had a leader election.
>
> Cheers
>
> Tom
>
> On Tue, Jul 19, 2016 at 4:30 PM, Erick Erickson  
> wrote:
>> How many replicas per Solr JVM? And do you
>> see any OOM errors when you bounce a server?
>> And how patient are you being, because it can
>> take 3 minutes for a leaderless shard to decide
>> it needs to elect a leader.
>>
>> See SOLR-7280 and SOLR-7191 for the case
>> where lots of replicas are in the same JVM,
>> the tell-tale symptom is errors in the log as you
>> bring Solr up saying something like
>> "OutOfMemory error unable to create native thread"
>>
>> SOLR-7280 has patches for 6x and 7x, with a 5x one
>> being added momentarily.
>>
>> Best,
>> Erick
>>
>> On Tue, Jul 19, 2016 at 7:41 AM, Tom Evans  wrote:
>>> Hi all - problem with a SolrCloud 5.5.0, we have a node that has most
>>> of the collections on it marked as "Recovering" or "Recovery Failed".
>>> It attempts to recover from the leader, but the leader responds with:
>>>
>>> Error while trying to recover.
>>> core=iris_shard1_replica1:java.util.concurrent.ExecutionException:
>>> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
>>> Error from server at http://172.31.1.171:3/solr: We are not the
>>> leader
>>> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>>> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>>> at 
>>> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:596)
>>> at 
>>> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:353)
>>> at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:224)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at 
>>> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
>>> at 
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at 
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: 
>>> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
>>> Error from server at http://172.31.1.171:3/solr: We are not the
>>> leader
>>> at 
>>> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
>>> at 
>>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:284)
>>> at 
>>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:280)
>>> ... 5 more
>>>
>>> and recovery never occurs.
>>>
>>> Each collection in this state has plenty (10+) of active replicas, but
>>> stopping the server that is marked as the leader doesn't trigger a
>>> leader 

Re: Node not recovering, leader elections not occuring

[Tom Evans]

There are 11 collections, each only has one shard, and each node has
10 replicas (9 collections are on every node, 2 are just on one node).
We're not seeing any OOM errors on restart.

I think we're being patient waiting for the leader election to occur.
We stopped the troublesome "leader that is not the leader" server
about 15-20 minutes ago, but we still have not had a leader election.

Cheers

Tom

On Tue, Jul 19, 2016 at 4:30 PM, Erick Erickson  wrote:
> How many replicas per Solr JVM? And do you
> see any OOM errors when you bounce a server?
> And how patient are you being, because it can
> take 3 minutes for a leaderless shard to decide
> it needs to elect a leader.
>
> See SOLR-7280 and SOLR-7191 for the case
> where lots of replicas are in the same JVM,
> the tell-tale symptom is errors in the log as you
> bring Solr up saying something like
> "OutOfMemory error unable to create native thread"
>
> SOLR-7280 has patches for 6x and 7x, with a 5x one
> being added momentarily.
>
> Best,
> Erick
>
> On Tue, Jul 19, 2016 at 7:41 AM, Tom Evans  wrote:
>> Hi all - problem with a SolrCloud 5.5.0, we have a node that has most
>> of the collections on it marked as "Recovering" or "Recovery Failed".
>> It attempts to recover from the leader, but the leader responds with:
>>
>> Error while trying to recover.
>> core=iris_shard1_replica1:java.util.concurrent.ExecutionException:
>> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
>> Error from server at http://172.31.1.171:3/solr: We are not the
>> leader
>> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>> at 
>> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:596)
>> at 
>> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:353)
>> at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:224)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at 
>> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
>> at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: 
>> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
>> Error from server at http://172.31.1.171:3/solr: We are not the
>> leader
>> at 
>> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
>> at 
>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:284)
>> at 
>> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:280)
>> ... 5 more
>>
>> and recovery never occurs.
>>
>> Each collection in this state has plenty (10+) of active replicas, but
>> stopping the server that is marked as the leader doesn't trigger a
>> leader election amongst these replicas.
>>
>> REBALANCELEADERS did nothing.
>> FORCELEADER complains that there is already a leader.
>> FORCELEADER with the purported leader stopped took 45 seconds,
>> reported status of "0" (and no other message) and kept the down node
>> as the leader (!)
>> Deleting the failed collection from the failed node and re-adding it
>> has the same "Leader said I'm not the leader" error message.
>>
>> Any other ideas?
>>
>> Cheers
>>
>> Tom

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

[Erick Erickson]

First of all, killing with -9 is A Very Bad Idea. You can
leave write lock files laying around. You can leave
the state in an "interesting" place. You haven't given
Solr a chance to tell Zookeeper that it's going away.
(which would set the state to "down"). In short
when you do this you have to deal with the consequences
yourself, one of which is this mismatch between
cluster state and live_nodes.

Now, that rant done the bin/solr script tries to stop Solr
gracefully but issues a kill if solr doesn't stop nicely. Personally
I think that timeout should be longer, but that's another story.

The onlyIfDown='true' option is there specifically as a
safety valve. It was provided for those who want to guard against
typos and the like, so just don't specify it and you should be fine.

Best,
Erick

On Mon, Jul 18, 2016 at 11:51 PM, Jerome Yang  wrote:
> Hi all,
>
> Here's the situation.
> I'm using solr5.3 in cloud mode.
>
> I have 4 nodes.
>
> After use "kill -9 pid-solr-node" to kill 2 nodes.
> These replicas in the two nodes still are "ACTIVE" in zookeeper's
> state.json.
>
> The problem is, when I try to delete these down replicas with
> parameter onlyIfDown='true'.
> It says,
> "Delete replica failed: Attempted to remove replica :
> demo.public.tbl/shard0/core_node4 with onlyIfDown='true', but state is
> 'active'."
>
> From this link:
> 
> 
> 
> 
> http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE
>
> It says:
> *NOTE*: when the node the replica is hosted on crashes, the replica's state
> may remain ACTIVE in ZK. To determine if the replica is truly active, you
> must also verify that its node
> 
> is
> under /live_nodes in ZK (or use ClusterState.liveNodesContain(String)
> 
> ).
>
> So, is this a bug?
>
> Regards,
> Jerome

config question (UNCLASSIFIED)

[Musshorn, Kris T CTR USARMY RDECOM ARL (US)]

CLASSIFICATION: UNCLASSIFIED

SOLR 6.1.0 + Nutch 2.3.1 works?

Thanks,
Kris

~~
Kris T. Musshorn
FileMaker Developer - Contractor - Catapult Technology Inc.  
US Army Research Lab 
Aberdeen Proving Ground 
Application Management & Development Branch 
410-278-7251
kris.t.musshorn@mail.mil
~~


CLASSIFICATION: UNCLASSIFIED

Re: Cold replication

[Erick Erickson]

The fact that your index is 200G is meaningless,
assuming you're talking about disk size. Please
just measure before you make assumptions about
what will work, it'll save you a world of hurt. I'm not
claiming that just using EBS will satisfy your
need, but if you're swapping your search speed will
suffer a lot, it doesn't matter whether you're swapping
to SSD or EBS. Well, it does matter but either way I'm
90% sure you won't be satisfied with performance. It's
just that you'll be _less_ unhappy with SSD.

If your index is changing rapidly, SSDs can be really
useful as they make loading parts of the index
into memory faster.

I say that the disk size of your index is meaningless, and
by that I mean that, for instance, if you've set stored="true"
for a field, a verbatim copy of that is stored on disk
and is largely irrelevant in terms of the memory it requires as
it's only read for assembling the final doc to return to
the user, not for finding matches. The stored data is held in
*.fdt files. The *.fdt files may be a very small percent of the
disk space or a very large percent, there's no way to know.
Other options also have disk .vs. memory implications.

As far as autowarming, that's simply a way to replay some
of the recent queries and filter queries when a new searcher
is opened (i.e. you commit new documents). It's intended
to smooth over spikes by moving relevant parts of the index
to memory from disk.

Best,
Erick


On Tue, Jul 19, 2016 at 1:16 AM, Emir Arnautovic
 wrote:
> Hi Mahmoud,
> What you can do is use local SSD disk as cache for EBS. You can try lvmcache
> or bcache. It will boost your performance while data will remain on EBS.
>
> Thanks,
> Emir
>
> --
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
>
>
>
>
> On 18.07.2016 19:34, Mahmoud Almokadem wrote:
>>
>> Thanks Erick,
>>
>> I'll take a look at the replication on Solr. But I don't know if it well
>> support incremental backup or not.
>>
>> And I want to use SSD because my index cannot be held in memory. The index
>> is about 200GB on each instance and the RAM is 61GB and the update
>> frequency is high. So, I want to use SSDs equipped with the servers
>> instead
>> on EBSs.
>>
>> Would you explain what you mean with proper warming?
>>
>> Thanks,
>> Mahmoud
>>
>>
>> On Mon, Jul 18, 2016 at 5:46 PM, Erick Erickson 
>> wrote:
>>
>>> Have you tried the replication API backup command here?
>>>
>>>
>>> https://cwiki.apache.org/confluence/display/solr/Index+Replication#IndexReplication-HTTPAPICommandsfortheReplicationHandler
>>>
>>> Warning, I haven't worked with this personally in this
>>> situation so test.
>>>
>>> I do have to ask why you think SSDs are required here and
>>> if you've measured. With proper warming, most of the
>>> index is held in memory anyway and the source of
>>> the data (SSD or spinning) is not a huge issue. SSDs
>>> certainly are better/faster, but have you measured whether
>>> they are _enough_ faster to be worth the added
>>> complexity?
>>>
>>> Best,
>>> Erick
>>>
>>> Best,
>>> Erick
>>>
>>> On Mon, Jul 18, 2016 at 4:05 AM, Mahmoud Almokadem
>>>  wrote:

 Hi,

 We have SolrCloud 6.0 installed on 4 i2.2xlarge instances with 4 shards.
>>>
>>> We store the indices on EBS attached to these instances. Fortunately
>>> these
>>> instances are equipped with TEMPORARY SSDs. We need to the store the
>>> indices on the SSDs but they are not safe.

 The index is updated every five minutes.

 Could we use the SSDs to store the indices and create an incremental
>>>
>>> backup or cold replication on the EBS? So we use EBS only for storing
>>> indices not serving the data to the solr.

 Incase of losing the data on SSDs we can restore a backup from the EBS.
>>>
>>> Is it possible?

 Thanks,
 Mahmoud


>

Re: SolrCloud - Query performance degrades with multiple servers(Shards)

[Erick Erickson]

15M docs may still comfortably fit in a single shard!
I've seen up to 300M docs fit on a shard. Then
again I've seen 10M docs make things unacceptably
slow.

You simply cannot extrapolate from 10K to
5M reliably. Put all 5M docs on the stand-alone
servers and test _that_. Whenever I see numbers
like 30K qps (assuming this is queries, not number
of docs indexed) I wonder if you're using the
same query over and over and hitting the query
result cache rather than doing any actual
searches.

But to answer your question (again). Sharding adds
overhead. There's no way to make that overhead
magically disappear. What you measure is what
you can expect, and you must measure.

Best,
Erick

On Tue, Jul 19, 2016 at 8:32 AM, Susheel Kumar  wrote:
> You may want to utilise Document routing (_route_) option to have your
> query serve faster but above you are trying to compare apple with oranges
> meaning your performance tests numbers have to be based on either your
> actual numbers like 3-5 million docs per shard or sufficient enough to see
> advantage of using sharding.  10K is nothing for your performance tests and
> will not give you anything.
>
> Otherwise as Eric mentioned don't shard  and add replica's if there is no
> need to distribute/divide data into shards.
>
>
> See
> https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+in+SolrCloud
>
> https://cwiki.apache.org/confluence/display/solr/Advanced+Distributed+Request+Options
>
>
> Thanks,
> Susheel
>
> On Tue, Jul 19, 2016 at 1:41 AM, kasimjinwala 
> wrote:
>
>> This is just for performance testing we have taken 10K records per shard.
>> In
>> live scenario it would be 30L-50L per shard. I want to search document from
>> all shards, it will slow down and take too long time.
>>
>> I know in case of solr Cloud, it will query all shard node and then return
>> result. Is there any way to search document in all shard with best
>> performance(qps)
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/SolrCloud-Query-performance-degrades-with-multiple-servers-tp4024660p4287763.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>

Re: SolrCloud - Query performance degrades with multiple servers(Shards)

[Susheel Kumar]

You may want to utilise Document routing (_route_) option to have your
query serve faster but above you are trying to compare apple with oranges
meaning your performance tests numbers have to be based on either your
actual numbers like 3-5 million docs per shard or sufficient enough to see
advantage of using sharding.  10K is nothing for your performance tests and
will not give you anything.

Otherwise as Eric mentioned don't shard  and add replica's if there is no
need to distribute/divide data into shards.


See
https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+in+SolrCloud

https://cwiki.apache.org/confluence/display/solr/Advanced+Distributed+Request+Options


Thanks,
Susheel

On Tue, Jul 19, 2016 at 1:41 AM, kasimjinwala 
wrote:

> This is just for performance testing we have taken 10K records per shard.
> In
> live scenario it would be 30L-50L per shard. I want to search document from
> all shards, it will slow down and take too long time.
>
> I know in case of solr Cloud, it will query all shard node and then return
> result. Is there any way to search document in all shard with best
> performance(qps)
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/SolrCloud-Query-performance-degrades-with-multiple-servers-tp4024660p4287763.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Node not recovering, leader elections not occuring

[Erick Erickson]

How many replicas per Solr JVM? And do you
see any OOM errors when you bounce a server?
And how patient are you being, because it can
take 3 minutes for a leaderless shard to decide
it needs to elect a leader.

See SOLR-7280 and SOLR-7191 for the case
where lots of replicas are in the same JVM,
the tell-tale symptom is errors in the log as you
bring Solr up saying something like
"OutOfMemory error unable to create native thread"

SOLR-7280 has patches for 6x and 7x, with a 5x one
being added momentarily.

Best,
Erick

On Tue, Jul 19, 2016 at 7:41 AM, Tom Evans  wrote:
> Hi all - problem with a SolrCloud 5.5.0, we have a node that has most
> of the collections on it marked as "Recovering" or "Recovery Failed".
> It attempts to recover from the leader, but the leader responds with:
>
> Error while trying to recover.
> core=iris_shard1_replica1:java.util.concurrent.ExecutionException:
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
> Error from server at http://172.31.1.171:3/solr: We are not the
> leader
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at 
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:596)
> at 
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:353)
> at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:224)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
> Error from server at http://172.31.1.171:3/solr: We are not the
> leader
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:284)
> at 
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:280)
> ... 5 more
>
> and recovery never occurs.
>
> Each collection in this state has plenty (10+) of active replicas, but
> stopping the server that is marked as the leader doesn't trigger a
> leader election amongst these replicas.
>
> REBALANCELEADERS did nothing.
> FORCELEADER complains that there is already a leader.
> FORCELEADER with the purported leader stopped took 45 seconds,
> reported status of "0" (and no other message) and kept the down node
> as the leader (!)
> Deleting the failed collection from the failed node and re-adding it
> has the same "Leader said I'm not the leader" error message.
>
> Any other ideas?
>
> Cheers
>
> Tom

Re: Parallel SQL where exists, in

[Erick Erickson]

There is a lot of activity in the ParallelSQL world, all being done
by a very few people, so it's a matter of priorities. Can you
consider submitting a patch?

Best,
Erick

On Tue, Jul 19, 2016 at 8:12 AM, Pablo Anzorena  wrote:
> Hey,
>
> Is anyone willing to add the where exists and in clauses into paraller sql?
>
> Thanks.

Solr DIH

[halis Yılboğa]

Cloud somebody help me to figure out my problem described below.

http://stackoverflow.com/questions/37946150/could-not-chain-dataimporthandler-and-schemaless-to-add-unknown-fields/37970809#37970809

Parallel SQL where exists, in

[Pablo Anzorena]

Hey,

Is anyone willing to add the where exists and in clauses into paraller sql?

Thanks.

Node not recovering, leader elections not occuring

[Tom Evans]

Hi all - problem with a SolrCloud 5.5.0, we have a node that has most
of the collections on it marked as "Recovering" or "Recovery Failed".
It attempts to recover from the leader, but the leader responds with:

Error while trying to recover.
core=iris_shard1_replica1:java.util.concurrent.ExecutionException:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://172.31.1.171:3/solr: We are not the
leader
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at 
org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:596)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:353)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:224)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://172.31.1.171:3/solr: We are not the
leader
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:576)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:284)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:280)
... 5 more

and recovery never occurs.

Each collection in this state has plenty (10+) of active replicas, but
stopping the server that is marked as the leader doesn't trigger a
leader election amongst these replicas.

REBALANCELEADERS did nothing.
FORCELEADER complains that there is already a leader.
FORCELEADER with the purported leader stopped took 45 seconds,
reported status of "0" (and no other message) and kept the down node
as the leader (!)
Deleting the failed collection from the failed node and re-adding it
has the same "Leader said I'm not the leader" error message.

Any other ideas?

Cheers

Tom

AW: AW: group.facet=true and facet on field of type int -> org.apache.solr.common.SolrException: Exception during facet.field

[Sebastian Riemer]

Hi Tomás!

Many thanks for responding - I agree, I'd say 
https://issues.apache.org/jira/browse/SOLR-7495 is definitely  the same issue. 
I am working around that issue by using a STR-Field and copyField.

Thanks again,

Sebastian

-Ursprüngliche Nachricht-
Von: Tomás Fernández Löbbe [mailto:tomasflo...@gmail.com] 
Gesendet: Dienstag, 19. Juli 2016 12:59
An: solr-user@lucene.apache.org
Betreff: Re: AW: group.facet=true and facet on field of type int -> 
org.apache.solr.common.SolrException: Exception during facet.field

Hi Sebastian,
This looks like https://issues.apache.org/jira/browse/SOLR-7495

On Jul 19, 2016 3:46 AM, "Sebastian Riemer"  wrote:

> May I respectfully refer again to a question I posted last week?
>
> Thank you very much and a nice day to you all!
>
> Sebastian
> -
>
>
>
>
>
>
>
> Hi all,
>
> Tested on Solr 6.1.0 (as well as 5.4.0 and 5.5.0) using the "techproducts"
> example the following query throws the same exception as in my 
> original
> question:
>
> To reproduce:
> 1) set up the techproducts example:
> solr start -e techproducts -noprompt
> 2) go to Solr Admin:
> http://localhost:8983/solr/#/techproducts/query
> 3) in "Raw Query Parameters" enter:
>
> group=true=true=true=manu_id_s
> acet=true=popularity
> 4) Hit "Execute Query"
>
> [..]
> "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","java.lang.IllegalStateException"],
> "msg":"Exception during facet.field: popularity",
> "trace":"org.apache.solr.common.SolrException: Exception during
> facet.field: popularity\r\n\tat
> org.apache.solr.request.SimpleFacets.lambda$getFacetFieldCounts$50(Sim
> pleFacets.java:739)\r\n\tat 
> org.apache.solr.request.SimpleFacets$$Lambda$37/2022187546.call(Unknow
> n
> Source)\r\n\tat
> java.util.concurrent.FutureTask.run(FutureTask.java:266)\r\n\tat
> org.apache.solr.request.SimpleFacets$2.execute(SimpleFacets.java:672)\
> r\n\tat 
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.
> java:748)\r\n\tat 
> org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetC
> omponent.java:321)\r\n\tat 
> org.apache.solr.handler.component.FacetComponent.process(FacetComponen
> t.java:265)\r\n\tat 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(Sear
> chHandler.java:293)\r\n\tat 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandle
> rBase.java:156)\r\n\tat 
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)\r\n\tat
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)\r\
> n\tat 
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)\r\n\t
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter
> .java:257)\r\n\tat 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter
> .java:208)\r\n\tat 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletH
> andler.java:1668)\r\n\tat 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:
> 581)\r\n\tat 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.ja
> va:143)\r\n\tat 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java
> :548)\r\n\tat 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandle
> r.java:226)\r\n\tat 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandle
> r.java:1160)\r\n\tat 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:5
> 11)\r\n\tat 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler
> .java:185)\r\n\tat 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler
> .java:1092)\r\n\tat 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.ja
> va:141)\r\n\tat 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(Conte
> xtHandlerCollection.java:213)\r\n\tat
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerColle
> ction.java:119)\r\n\tat 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.
> java:134)\r\n\tat 
> org.eclipse.jetty.server.Server.handle(Server.java:518)\r\n\tat
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)\r\n\
> tat 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java
> :244)\r\n\tat 
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(Abstrac
> tConnection.java:273)\r\n\tat 
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\r\n\t
> at 
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint
> .java:93)\r\n\tat 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAn
> dRun(ExecuteProduceConsume.java:246)\r\n\tat
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(Execu
> teProduceConsume.java:156)\r\n\tat
> 

Re: index sql databases

[Shawn Heisey]

On 7/19/2016 4:56 AM, kostali hassan wrote:
> I am looking for display for each user: l'utilisateur est crée le
> $date à $time not $document-> name est crée le $document->created 

You'll have to do that in your application that queries Solr, splitting
the date and time information that it receives in one field, or you'll
have to write a plugin for Solr that overrides the normal output into
two fields.  The first option is likely much easier.

Thanks,
Shawn

Re: Solr Cloud - how to implement local indexing without SSL and distributed search with SSL

[Sarit Weber]

Hi again,

Do you think it's possible to do that with server that will be dedicate to 
indexing and server that will be dedicate to search but will work on the 
same collections?

Thanks,
Sarit Weber 
Guardium Software Developer
IBM Israel Software Labs, Jerusalem
Phone: +972-2-649-1712
email:  sar...@il.ibm.com 





From:   Sarit Weber/Haifa/IBM@IBMIL
To: solr-user@lucene.apache.org
Date:   17/07/2016 05:21 PM
Subject:Solr Cloud - how to implement local indexing without SSL 
and distributed search with SSL



Hi all,
 
We are currently using Solr 6.0.0, with Solr Cloud and SSL 
where:
1. Collections are defined with router.name=implicit 
2. Collections are built of shard per machine 
3. Data from a specific machine is indexed on that shard and documents 
always keep on coming. 
4. Search is distributed - from each machine we want to search for data on 

the entire collection - using the Solr Cloud API. 
 
We noticed that indexing is much faster without SSL, but we can not remove 

it from distributed search. 
 
Because we force the index to be local (each machine index its own data) 
- 
We were wondering if there is an option to remove SSL from indexing and 
keep using it for Searching.
The solution will have to require the indexing to be done locally, not 
calling the remote zookeeper. 
 
Is there any way to achieve this with Solr Cloud?

Thanks,
Sarit Weber 

Re: AW: group.facet=true and facet on field of type int -> org.apache.solr.common.SolrException: Exception during facet.field

[Tomás Fernández Löbbe]

Hi Sebastian,
This looks like https://issues.apache.org/jira/browse/SOLR-7495

On Jul 19, 2016 3:46 AM, "Sebastian Riemer"  wrote:

> May I respectfully refer again to a question I posted last week?
>
> Thank you very much and a nice day to you all!
>
> Sebastian
> -
>
>
>
>
>
>
>
> Hi all,
>
> Tested on Solr 6.1.0 (as well as 5.4.0 and 5.5.0) using the "techproducts"
> example the following query throws the same exception as in my original
> question:
>
> To reproduce:
> 1) set up the techproducts example:
> solr start -e techproducts -noprompt
> 2) go to Solr Admin:
> http://localhost:8983/solr/#/techproducts/query
> 3) in "Raw Query Parameters" enter:
>
> group=true=true=true=manu_id_s=true=popularity
> 4) Hit "Execute Query"
>
> [..]
> "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","java.lang.IllegalStateException"],
> "msg":"Exception during facet.field: popularity",
> "trace":"org.apache.solr.common.SolrException: Exception during
> facet.field: popularity\r\n\tat
> org.apache.solr.request.SimpleFacets.lambda$getFacetFieldCounts$50(SimpleFacets.java:739)\r\n\tat
> org.apache.solr.request.SimpleFacets$$Lambda$37/2022187546.call(Unknown
> Source)\r\n\tat
> java.util.concurrent.FutureTask.run(FutureTask.java:266)\r\n\tat
> org.apache.solr.request.SimpleFacets$2.execute(SimpleFacets.java:672)\r\n\tat
> org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:748)\r\n\tat
> org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:321)\r\n\tat
> org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:265)\r\n\tat
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:293)\r\n\tat
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)\r\n\tat
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)\r\n\tat
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)\r\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)\r\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)\r\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)\r\n\tat
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)\r\n\tat
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)\r\n\tat
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\r\n\tat
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\r\n\tat
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\r\n\tat
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)\r\n\tat
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)\r\n\tat
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\r\n\tat
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)\r\n\tat
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\r\n\tat
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\r\n\tat
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\r\n\tat
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\r\n\tat
> org.eclipse.jetty.server.Server.handle(Server.java:518)\r\n\tat
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)\r\n\tat
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)\r\n\tat
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\r\n\tat
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\r\n\tat
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\r\n\tat
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)\r\n\tat
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)\r\n\tat
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)\r\n\tat
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)\r\n\tat
> java.lang.Thread.run(Thread.java:745)\r\nCaused by:
> java.lang.IllegalStateException: unexpected docvalues type NUMERIC for
> field 'popularity' (expected=SORTED). Use UninvertingReader or index with
> docvalues.\r\n\tat
> org.apache.lucene.index.DocValues.checkField(DocValues.java:212)\r\n\tat
> org.apache.lucene.index.DocValues.getSorted(DocValues.java:264)\r\n\tat
> org.apache.lucene.search.grouping.term.TermGroupFacetCollector$SV.doSetNextReader(TermGroupFacetCollector.java:129)\r\n\tat
> 

Re: index sql databases

[kostali hassan]

I am looking for display for each user:
l'utilisateur est crée le $date à $time
not
$document-> name est crée le $document->created

2016-07-18 16:48 GMT+01:00 Erick Erickson :

> I don't see how that relates to the original
> question.
>
> bq: when I display the field type date I get the
> value in this forme -MM-dd'T'hh:mm:ss'Z'
>
> A regex in the _input_ side will have
> no effect on what Solr returns. You'd have
> to use a DocTransformer to change the output
> on the query side. DIH is in the indexing side.
>
> Best,
> Erick
>
> On Mon, Jul 18, 2016 at 2:45 AM, kostali hassan
>  wrote:
> > can we use transformer="RegexTransformer"
> > and set in db_data_config.xml
> >   > groupNames="date_t,time_t" />
> >
> > 2016-07-16 18:18 GMT+01:00 Shawn Heisey :
> >
> >> On 7/15/2016 3:10 PM, kostali hassan wrote:
> >> > Thank you Shawn the prb is when I display the field type date I get
> >> > the value in this forme -MM-dd'T'hh:mm:ss'Z'
> >>
> >> Solr only displays ISO date format for date fields -- an example is
> >> 2016-07-16T18:17:08.497Z -- and only in the UTC timezone.  If you want
> >> something else in your application, you'll have to translate it, or
> >> you'll have to write a custom plugin to add to Solr that changes the
> >> output format.
> >>
> >> Thanks,
> >> Shawn
> >>
> >>
>

More like this in solr5.4.1

[kostali hassan]

I want introdius Morelikethis to get simmilaire document for each query.
I had index rich data pds and msword I guess The fields to use for
similarity
is CONTENT used also for  highlighting document content.
In my case what is the best way to build mlt :MoreLikeThisHandler
 or with the
MoreLikeThisComponent in SearchHandler
.

Re: SolrCloud - Query performance degrades with multiple servers(Shards)

[kasimjinwala]

This is just for performance testing we have taken 10K records per shard. In
live scenario it would be 30L-50L per shard. I want to search document from
all shards, it will slow down and take too long time. 

I know in case of solr Cloud, it will query all shard node and then return
result. Is there any way to search document in all shard with best
performance(qps)



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SolrCloud-Query-performance-degrades-with-multiple-servers-tp4024660p4287763.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Cold replication

[Emir Arnautovic]

Hi Mahmoud,
What you can do is use local SSD disk as cache for EBS. You can try 
lvmcache or bcache. It will boost your performance while data will 
remain on EBS.


Thanks,
Emir

--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/



On 18.07.2016 19:34, Mahmoud Almokadem wrote:

Thanks Erick,

I'll take a look at the replication on Solr. But I don't know if it well
support incremental backup or not.

And I want to use SSD because my index cannot be held in memory. The index
is about 200GB on each instance and the RAM is 61GB and the update
frequency is high. So, I want to use SSDs equipped with the servers instead
on EBSs.

Would you explain what you mean with proper warming?

Thanks,
Mahmoud


On Mon, Jul 18, 2016 at 5:46 PM, Erick Erickson 
wrote:


Have you tried the replication API backup command here?

https://cwiki.apache.org/confluence/display/solr/Index+Replication#IndexReplication-HTTPAPICommandsfortheReplicationHandler

Warning, I haven't worked with this personally in this
situation so test.

I do have to ask why you think SSDs are required here and
if you've measured. With proper warming, most of the
index is held in memory anyway and the source of
the data (SSD or spinning) is not a huge issue. SSDs
certainly are better/faster, but have you measured whether
they are _enough_ faster to be worth the added
complexity?

Best,
Erick

Best,
Erick

On Mon, Jul 18, 2016 at 4:05 AM, Mahmoud Almokadem
 wrote:

Hi,

We have SolrCloud 6.0 installed on 4 i2.2xlarge instances with 4 shards.

We store the indices on EBS attached to these instances. Fortunately these
instances are equipped with TEMPORARY SSDs. We need to the store the
indices on the SSDs but they are not safe.

The index is updated every five minutes.

Could we use the SSDs to store the indices and create an incremental

backup or cold replication on the EBS? So we use EBS only for storing
indices not serving the data to the solr.

Incase of losing the data on SSDs we can restore a backup from the EBS.

Is it possible?

Thanks,
Mahmoud

Re: SOLR Suggester | CFQ Variable | - not supported

[Rajesh Kapur]

Hi,

Any update on this please.

Thanks.

On Sun, Jul 17, 2016 at 9:25 PM, Rajesh Kapur 
wrote:

> Hi,
>
> Thanks for the reply.
>
> Yes, i tried to search by setting CFQ=abc\-def and also as "abc-def", but
> no luck.
>
> Thanks.
>
> On Sun, Jul 17, 2016 at 9:19 PM, Erick Erickson 
> wrote:
>
>> What have you tried? Did you try escaping it?
>>
>> Best,
>> Erick
>>
>> On Sun, Jul 17, 2016 at 7:32 AM, Rajesh Kapur 
>> wrote:
>> > Hi
>> >
>> > I am still facing this issue. Anyone can help me in this.
>> >
>> > Thanks
>> >
>> > On 14-Jul-2016 1:15 PM, "Rajesh Kapur"  wrote:
>> >
>> >> Hi,
>> >>
>> >> I am facing issue while implementing suggester for my project where I
>> am
>> >> passing CFQ value having - in between, but it is not giving me desired
>> >> output.
>> >>
>> >> Could you please let me know what should I do so that SOLR suggester
>> >> starts accepting - in CFQ parameters.
>> >>
>> >> Thanks
>> >> Rajesh Kapur
>> >>
>>
>
>

Re: Problem using bbox in schema

[Rastislav Hudak]

Sure, here it is:












_id









































































  

  



  



  
  




  



  






  
  







  



  





  



  





  




Cheers!
Rasta

On 18 July 2016 at 23:49, Chris Hostetter  wrote:

>
> can you please send us the entire schema.xml file you were using when you
> got that error?
>
> Would be nice to get to the bottom of how/why you gut an error regarding
> "positionIncrementGap" on AbstractSpatialPrefixTreeFieldType.
>
>
>
> : Date: Mon, 18 Jul 2016 18:10:40 +0200
> : From: Rastislav Hudak 
> : Reply-To: solr-user@lucene.apache.org
> : To: solr-user@lucene.apache.org
> : Subject: Re: Problem using bbox in schema
> :
> : Thanks Erick.
> : I wasn't able to track this down but when I re-set the schema to the
> sample
> : one and added the bbox it was working. I thought it must be the bbox
> field
> : because of that 'AbstractSpatialPrefixTreeFieldType' but it must have
> been
> : something else. The only difference I can see now is that I left all the
> : sample fieldtypes in, eg previously I deleted "point" as I didn't need
> it. So
> : maybe there's some dependency I missed.
> :
> : Thanks anyways!
> :
> : Rasta
> :
> : On 2016-07-18 17:56, Erick Erickson wrote:
> : > Those work fine for me (latest 6x version). It looks to me like
> : > you've changed some _other_ field, this error
> : >
> : > Can't set positionIncrementGap on custom analyzer class
> : >
> : > looks like you have some text-related field you've also changed
> : > and likely have a custom analyzer somewhere?
> : >
> : > Best,
> : > Erick
> : >
> : > On Mon, Jul 18, 2016 at 1:26 AM, Rastislav Hudak
> : >  wrote:
> : > > Hi all,
> : > >
> : > > using solr 6.1.0, I'm trying to add bbox into my schema, the same
> way it's
> : > > in all examples I could find:
> : > >
> : > > 
> : > >  : > > numberType="_bbox_coord" storeSubFields="false" />
> : > >  : > > precisionStep="8" docValues="true" stored="false"/>
> : > >
> : > > However, if I add these lines the core cannot be initialized, I'm
> getting
> : > > following exception:
> : > >
> : > > java.util.concurrent.ExecutionException:
> : > > org.apache.solr.common.SolrException: Unable to create core [phaidra]
> : > > at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> : > > at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> : > > at org.apache.solr.core.CoreContainer$1.run(CoreContainer.java:494)
> : > > at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> : > > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> : > > at
> : > >
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$22(ExecutorUtil.java:229)
> : > > at
> : > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> : > > at
> : > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> : > > at java.lang.Thread.run(Thread.java:745)
> : > > Caused by: org.apache.solr.common.SolrException: Unable to create
> core
> : > > [phaidra]
> : > > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:825)
> : > > at
> : > >
> org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:466)
> : > > ... 5 more
> : > > Caused by: org.apache.solr.common.SolrException: Could not load conf
> for
> : > > core phaidra: Can't load schema
> : > > /var/solr/data/phaidra/conf/managed-schema:
> : > > Plugin Initializing failure for [schema.xml] fieldType
> : > > at
> : > >
> org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:86)
> : > > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:810)
> : > > ... 6 more
> : > > Caused by: org.apache.solr.common.SolrException: Can't load schema
> : > > /var/solr/data/phaidra/conf/managed-schema: Plugin Initializing
> failure
> : > > for
> : > > [schema.xml] fieldType
> : > > at
> org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:600)
> : > > at org.apache.solr.schema.IndexSchema.(IndexSchema.java:183)
> : > > at
> : > >
> org.apache.solr.schema.ManagedIndexSchema.(ManagedIndexSchema.java:104)
> : > > at
> : > >
> org.apache.solr.schema.ManagedIndexSchemaFactory.create(ManagedIndexSchemaFactory.java:174)
> : > > at
> : > >
> 

Send kill -9 to a node and can not delete down replicas with onlyIfDown.

[Jerome Yang]

Hi all,

Here's the situation.
I'm using solr5.3 in cloud mode.

I have 4 nodes.

After use "kill -9 pid-solr-node" to kill 2 nodes.
These replicas in the two nodes still are "ACTIVE" in zookeeper's
state.json.

The problem is, when I try to delete these down replicas with
parameter onlyIfDown='true'.
It says,
"Delete replica failed: Attempted to remove replica :
demo.public.tbl/shard0/core_node4 with onlyIfDown='true', but state is
'active'."

>From this link:




http://www.solr-start.com/javadoc/solr-lucene/org/apache/solr/common/cloud/Replica.State.html#ACTIVE

It says:
*NOTE*: when the node the replica is hosted on crashes, the replica's state
may remain ACTIVE in ZK. To determine if the replica is truly active, you
must also verify that its node

is
under /live_nodes in ZK (or use ClusterState.liveNodesContain(String)

).

So, is this a bug?

Regards,
Jerome

AW: group.facet=true and facet on field of type int -> org.apache.solr.common.SolrException: Exception during facet.field

[Sebastian Riemer]

May I respectfully refer again to a question I posted last week?

Thank you very much and a nice day to you all!

Sebastian
-







Hi all,

Tested on Solr 6.1.0 (as well as 5.4.0 and 5.5.0) using the "techproducts" 
example the following query throws the same exception as in my original 
question:

To reproduce:
1) set up the techproducts example: 
solr start -e techproducts -noprompt
2) go to Solr Admin: 
http://localhost:8983/solr/#/techproducts/query
3) in "Raw Query Parameters" enter: 

group=true=true=true=manu_id_s=true=popularity
4) Hit "Execute Query"

[..]
"error":{
"metadata":[
  "error-class","org.apache.solr.common.SolrException",
  "root-error-class","java.lang.IllegalStateException"],
"msg":"Exception during facet.field: popularity",
"trace":"org.apache.solr.common.SolrException: Exception during 
facet.field: popularity\r\n\tat 
org.apache.solr.request.SimpleFacets.lambda$getFacetFieldCounts$50(SimpleFacets.java:739)\r\n\tat
 org.apache.solr.request.SimpleFacets$$Lambda$37/2022187546.call(Unknown 
Source)\r\n\tat 
java.util.concurrent.FutureTask.run(FutureTask.java:266)\r\n\tat 
org.apache.solr.request.SimpleFacets$2.execute(SimpleFacets.java:672)\r\n\tat 
org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:748)\r\n\tat
 
org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:321)\r\n\tat
 
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:265)\r\n\tat
 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:293)\r\n\tat
 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)\r\n\tat
 org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)\r\n\tat 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)\r\n\tat 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)\r\n\tat 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)\r\n\tat
 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)\r\n\tat
 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)\r\n\tat
 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)\r\n\tat
 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\r\n\tat
 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\r\n\tat
 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\r\n\tat
 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)\r\n\tat
 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)\r\n\tat
 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\r\n\tat
 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)\r\n\tat
 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\r\n\tat
 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\r\n\tat
 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\r\n\tat
 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\r\n\tat
 org.eclipse.jetty.server.Server.handle(Server.java:518)\r\n\tat 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)\r\n\tat 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)\r\n\tat
 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\r\n\tat
 org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\r\n\tat 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\r\n\tat
 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)\r\n\tat
 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)\r\n\tat
 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)\r\n\tat
 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)\r\n\tat
 java.lang.Thread.run(Thread.java:745)\r\nCaused by: 
java.lang.IllegalStateException: unexpected docvalues type NUMERIC for field 
'popularity' (expected=SORTED). Use UninvertingReader or index with 
docvalues.\r\n\tat 
org.apache.lucene.index.DocValues.checkField(DocValues.java:212)\r\n\tat 
org.apache.lucene.index.DocValues.getSorted(DocValues.java:264)\r\n\tat 
org.apache.lucene.search.grouping.term.TermGroupFacetCollector$SV.doSetNextReader(TermGroupFacetCollector.java:129)\r\n\tat
 
org.apache.lucene.search.SimpleCollector.getLeafCollector(SimpleCollector.java:33)\r\n\tat
 org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:660)\r\n\tat 
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)\r\n\tat