Re: Parent child documents partial update

[Sujay Bawaskar]

Thanks Amrit. So storage mechanism of parent child documents is limiting
the capability of partial update. It would be great to have flawless parent
child index support in solr.

On 17-Jul-2017 11:14 PM, "Amrit Sarkar"  wrote:

> Sujay,
>
> Not really. Parent-child documents are stored in a single block
> contiguously. Read more about parent-child relationship at:
> https://medium.com/@sarkaramrit2/multiple-documents-with-same-doc-id-in-
> index-in-solr-cloud-32c072db2164
>
> While we perform partial / atomic update, say {"id":"X",
> "fieldA":{"set":"Z"}, that particular doc with X will be fetched (all the
> "stored" fields), update will be performed and indexed, all happens in
> *DistributedUpdateProcessor* internally. So there is no way it will fetch
> the child documents along with it.
>
> I am not sure whether this can be done with current code or it will be
> fixed / improved in the future.
>
> Amrit Sarkar
> Search Engineer
> Lucidworks, Inc.
> 415-589-9269
> www.lucidworks.com
> Twitter http://twitter.com/lucidworks
> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>
> On Mon, Jul 17, 2017 at 12:44 PM, Sujay Bawaskar 
> wrote:
>
> > Hi,
> >
> > Need a help to understand solr parent child document partial update
> > behaviour. Can we perform partial update on parent document without
> losing
> > its chiild documents? My observation is that parent child relationship
> > between documents get lost in case partial update is performed on parent.
> > Any work around or solution to this issue?
> >
> > --
> > Thanks,
> > Sujay P Bawaskar
> > M:+91-77091 53669
> >
>

Re: Copy field a source of copy field

[Shawn Heisey]

On 7/17/2017 4:26 PM, tstusr wrote:
> We want to use a copy field as a source for another copy field or some kind
> of post processing of a field.

> As an example imagine we have on species
>
> abies durangensis
> abies flinckii
>
> so, after post processing, we expect to have only
> abies
>
> which is a word in genus files

Let's say that you have this in your schema, and you index "Test Words"
(note the capital letters) in field a:



Let's say that the index analysis on field a has the whitespace
tokenizer, a lowercase filter, and a stopword filter with "test" in the
list.  This means that the search terms for field a on that document
will only have "words" included.

You might be expecting field b to only receive "words" when it gets
copied from field a ... but this is NOT what happens.  Field b receives
the original text sent to field a, which is "Test Words", including both
words and the uppercase letters.

I think that transitive copies *do* work, so that you can copy field a
to b, then field b to c, though I am not 100 percent sure about that. 
If that does work, the end field in the chain is still going to receive
"Test Words" like you sent to field a.

Chaining analysis through copyField does not work.

Thanks,
Shawn

Re: Copy field a source of copy field

[Erick Erickson]

In a word, "no". Copyfields are not chained together. I'm not at all
sure what you're trying to accomplish with those filter chains anyway,
By shingling _then_ doing the stopwords, you'll have some input like
abies durangensis

become

abies
abies_durangensis
durangensis

Then put that through your keepwords filter which presumably only has
species in it so it would throw out abies and abies_durangensis unless
those are in your keepwords file Seems a waste.

That aside, you can construct one long analysis chain that combined
the genus and species chains and just copy from attr_content* into
both. You wouldn't get the different tokenization, but presumably you
don't particularly need it on the second part of the chain.

Best,
Erick

On Mon, Jul 17, 2017 at 3:26 PM, tstusr  wrote:
> Hi
>
> We want to use a copy field as a source for another copy field or some kind
> of post processing of a field.
>
> The problem is here. We have a field from a text that is captured by a
> field, like this:
>
> 
>
> which has (at the end of the processing) just the words in a field.
>
>  termVectors="true" termPositions="true" termOffsets="true"/>
>
>  positionIncrementGap="0">
> 
>   
>mapping="mapping/mapping-ISOLatin1Accent.txt"/>
>pattern="[0-9]+|(\-)(\s*)" replacement=""/>
>   
>outputUnigrams="true"/>
>ignoreCase="true"/>
> 
> 
>   
>outputUnigrams="false"/>
>   
> 
>   
>
> So, what we want to do now is to implement a faceting according to some post
> processing of this field by using this as a source for another field.
>
> 
>
>  positionIncrementGap="0">
> 
>   
>   
>ignoreCase="true"/>
> 
> 
>   
>   
> 
>   
>
>
> As soon as I understand. We don't have a value on genus because the chain is
> ended. Nevertheless, we are also not available to make two processings to
> first, capture the words on species and then make a new capture for the
> genus.
>
> As an example imagine we have on species
>
> abies durangensis
> abies flinckii
>
> so, after post processing, we expect to have only
> abies
>
> which is a word in genus files
>
> I was as clear as possible with the problem, but maybe there are some black
> holes in the explanation.
>
> Hope you can help me.
>
>
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Copy-field-a-source-of-copy-field-tp4346425.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Copy field a source of copy field

[tstusr]

Hi

We want to use a copy field as a source for another copy field or some kind
of post processing of a field.

The problem is here. We have a field from a text that is captured by a
field, like this:



which has (at the end of the processing) just the words in a field.





  
  
  
  
  
  


  
  
  

  

So, what we want to do now is to implement a faceting according to some post
processing of this field by using this as a source for another field.





  
  
  


  
  

  


As soon as I understand. We don't have a value on genus because the chain is
ended. Nevertheless, we are also not available to make two processings to
first, capture the words on species and then make a new capture for the
genus.

As an example imagine we have on species

abies durangensis
abies flinckii

so, after post processing, we expect to have only
abies

which is a word in genus files

I was as clear as possible with the problem, but maybe there are some black
holes in the explanation.

Hope you can help me.





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Copy-field-a-source-of-copy-field-tp4346425.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Limit to the number of cores supported?

[Erick Erickson]

I know of thousands of cores on a single Solr instance. Operationally
there's not problem there, although there may be some practical issues
(i.e. startup time and the like).

What does your Solr log show? Two popular issues:
OutOfMemory issues
Not enough file handles (fix with ulimit)

 But without more specific info about what Solr reports in the log
it's impossible to say much.

Best,
Erick

On Mon, Jul 17, 2017 at 10:41 AM, Pouliot, Scott
 wrote:
> Hey guys.
>
> We're running SOLR 6.2.0 in a master/slave configuration and I was wondering 
> if there is a limit to the number of cores this setup can support? We're 
> having random issue where a core or 2 will stop responding to POSTS (GETS 
> work fine) until we restart SOLR.
>
> We've currently got 140+ cores on this setup and wondering if that could be 
> part of the problem?
>
> Anyone ever run into this before?
>
> Scott

Re: CloudSolrClient preferred over LBHttpSolrClient

[Susheel Kumar]

Also per def of CloudSolrClient.


SolrJ client class to communicate with SolrCloud. Instances of this class
communicate with Zookeeper to discover Solr endpoints for SolrCloud
collections, and then use the LBHttpSolrClient

to
issue requests.

On Mon, Jul 17, 2017 at 3:15 PM, Amrit Sarkar 
wrote:

> S G,
>
> Not sure about the documentation but:
>
> The CloudSolrClient uses a connection to zookeeper to extract cluster
> information like who is a the leader for a shard in a solr collection. To
> create a CloudSolrClient all you specify is the zookeepers and which
> collection you want to work with. Behind the scenes solrj will load balance
> and send the request to the right "shard" in the cluster. The
> CloudSolrClient is better if you have a cluster of multiple solr nodes
> across multiple machines.
>
> While in LBHttpSolrClient, load balancing is done using a simple
> round-robin on the list of servers.
>
> Hope this helps.
>
> Amrit Sarkar
> Search Engineer
> Lucidworks, Inc.
> 415-589-9269
> www.lucidworks.com
> Twitter http://twitter.com/lucidworks
> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>
> On Mon, Jul 17, 2017 at 11:38 PM, S G  wrote:
>
> > Hi,
> >
> > Does anyone know if CloudSolrClient is preferred over LBHttpSolrClient ?
> > If yes, why so and has there been any good performance benefits
> documented
> > anywhere?
> >
> > Thanks
> > SG
> >
>

Re: CloudSolrClient preferred over LBHttpSolrClient

[Erick Erickson]

Also, since CloudSolrClient is ZK aware it is notified when any Solr
instances go up and down so it will take the appropriate action. Also,
when indexing CloudSolrClient will send updates to the correct leader,
reducing the hops for indexing documents.

Short form: CloudSolrClient is preferred over LBHttpSolrClient.

Best,
Erick

On Mon, Jul 17, 2017 at 12:15 PM, Amrit Sarkar  wrote:
> S G,
>
> Not sure about the documentation but:
>
> The CloudSolrClient uses a connection to zookeeper to extract cluster
> information like who is a the leader for a shard in a solr collection. To
> create a CloudSolrClient all you specify is the zookeepers and which
> collection you want to work with. Behind the scenes solrj will load balance
> and send the request to the right "shard" in the cluster. The
> CloudSolrClient is better if you have a cluster of multiple solr nodes
> across multiple machines.
>
> While in LBHttpSolrClient, load balancing is done using a simple
> round-robin on the list of servers.
>
> Hope this helps.
>
> Amrit Sarkar
> Search Engineer
> Lucidworks, Inc.
> 415-589-9269
> www.lucidworks.com
> Twitter http://twitter.com/lucidworks
> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>
> On Mon, Jul 17, 2017 at 11:38 PM, S G  wrote:
>
>> Hi,
>>
>> Does anyone know if CloudSolrClient is preferred over LBHttpSolrClient ?
>> If yes, why so and has there been any good performance benefits documented
>> anywhere?
>>
>> Thanks
>> SG
>>

Re: CloudSolrClient preferred over LBHttpSolrClient

[Amrit Sarkar]

S G,

Not sure about the documentation but:

The CloudSolrClient uses a connection to zookeeper to extract cluster
information like who is a the leader for a shard in a solr collection. To
create a CloudSolrClient all you specify is the zookeepers and which
collection you want to work with. Behind the scenes solrj will load balance
and send the request to the right "shard" in the cluster. The
CloudSolrClient is better if you have a cluster of multiple solr nodes
across multiple machines.

While in LBHttpSolrClient, load balancing is done using a simple
round-robin on the list of servers.

Hope this helps.

Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2

On Mon, Jul 17, 2017 at 11:38 PM, S G  wrote:

> Hi,
>
> Does anyone know if CloudSolrClient is preferred over LBHttpSolrClient ?
> If yes, why so and has there been any good performance benefits documented
> anywhere?
>
> Thanks
> SG
>

CloudSolrClient preferred over LBHttpSolrClient

[S G]

Hi,

Does anyone know if CloudSolrClient is preferred over LBHttpSolrClient ?
If yes, why so and has there been any good performance benefits documented
anywhere?

Thanks
SG

Re: Help with updateHandler commit stats

[Amrit Sarkar]

Antonio,

I think it is itself suggesting what it is. Meanwhile in official
documentation:

autocommits

Total number of auto-commits executed.

so yeah, total number of commits executed in the core's lifetime.

Look into:
https://cwiki.apache.org/confluence/display/solr/Performance+Statistics+Reference
for more details.

Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2

On Fri, Jul 7, 2017 at 4:15 PM, Antonio De Miguel 
wrote:

> Hi,
>
> I'm taking a look to UpdateHandler stats... and i see when autosoftcommit
> occurs (every 10 secs) both metrics, "commits" and "soft autocommits"
> increments by one. ¿is this normal?
>
> My config is:
>
> autoCommit: 180 secs
> autoSoftCommit: 10 secs
>
> Thanks!
>

Re: Parent child documents partial update

[Amrit Sarkar]

Sujay,

Not really. Parent-child documents are stored in a single block
contiguously. Read more about parent-child relationship at:
https://medium.com/@sarkaramrit2/multiple-documents-with-same-doc-id-in-index-in-solr-cloud-32c072db2164

While we perform partial / atomic update, say {"id":"X",
"fieldA":{"set":"Z"}, that particular doc with X will be fetched (all the
"stored" fields), update will be performed and indexed, all happens in
*DistributedUpdateProcessor* internally. So there is no way it will fetch
the child documents along with it.

I am not sure whether this can be done with current code or it will be
fixed / improved in the future.

Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2

On Mon, Jul 17, 2017 at 12:44 PM, Sujay Bawaskar 
wrote:

> Hi,
>
> Need a help to understand solr parent child document partial update
> behaviour. Can we perform partial update on parent document without losing
> its chiild documents? My observation is that parent child relationship
> between documents get lost in case partial update is performed on parent.
> Any work around or solution to this issue?
>
> --
> Thanks,
> Sujay P Bawaskar
> M:+91-77091 53669
>

Limit to the number of cores supported?

[Pouliot, Scott]

Hey guys.

We're running SOLR 6.2.0 in a master/slave configuration and I was wondering if 
there is a limit to the number of cores this setup can support? We're having 
random issue where a core or 2 will stop responding to POSTS (GETS work fine) 
until we restart SOLR.

We've currently got 140+ cores on this setup and wondering if that could be 
part of the problem?

Anyone ever run into this before?

Scott

Re: Solr 6.6.0 - Indexing errors

[Joe Obernberger]

We use puppet to deploy the solr instance to all the nodes.  I changed 
what was deployed to use the CDH jars, but our puppet module deletes the 
old directory and replaces it.  So, all the core configuration files 
under server/solr/ were removed. Zookeeper still has the configuration, 
but the nodes won't come up.


Is there a way around this?  Re-creating these files manually isn't 
realistic; do I need to re-index?


-Joe


On 7/17/2017 12:07 PM, Susheel Kumar wrote:

and there is document id mentioned above when it failed with analysis
error.  You can look how those documents differ as Eric suggested.

On Mon, Jul 17, 2017 at 11:53 AM, Erick Erickson 
wrote:


Joe:

I agree that 46 million docs later you'd expect things to have settled
out. However, I do note that you have
"add-unknown-fields-to-the-schema" in your error stack which means
you're using "field guessing", sometimes called data_driven. I would
recommend you do _not_ use this for production as, while it does the
best job it can it has to make assumptions about what the data looks
like based on the first document it sees which may later be violated.
Getting "possible analysis error" is one of the messages that happens
when this occurs.

The simple example is that if the first time data_driven sees "1"
it'll guess integer. If sometime later there's a doc with "1.0" it'll
generate a parse error.

I totally agree that 46 million docs later you'd expect all of this
kind of thing to have flushed out, but the "possible analysis error"
seems to be pointing that direction. If this is, indeed, the problem
you'll see better evidence on the Solr instance that's actually having
the problem. Unfortunately you'll just to look at one Solr log from
each shard to see whether this is an issue.

Best,
Erick

On Mon, Jul 17, 2017 at 7:23 AM, Joe Obernberger
 wrote:

So far we've indexed about 46 million documents, but over the weekend,

these

errors started coming up.  I would expect that if there was a basic

issue,

it would have started right away?  We ran a test cluster with just a few
shards/replicas prior and didn't see any issues using the same indexing
code, but we're running a lot more indexers simultaneously with the

larger

cluster; perhaps we're just overloading HDFS?  The same nodes that run

Solr

also run HDFS datanodes, but they are pretty beefy machines; we're not
swapping.

As Shawn pointed out, I will be checking the HDFS version (we're using
Cloudera CDH 5.10.2), and the HDFS logs.

-Joe



On 7/17/2017 10:16 AM, Susheel Kumar wrote:

There is some analysis error also.  I would suggest to test the indexer

on

just one shard setup first, then test for a replica (1 shard and 1
replica)
and then test for 2 shards and 2 replica.  This would confirm if there

is

basic issue with indexing / cluster setup.

On Mon, Jul 17, 2017 at 9:04 AM, Joe Obernberger <
joseph.obernber...@gmail.com> wrote:


Some more info:

When I stop all the indexers, in about 5-10 minutes the cluster goes

all

green.  When I start just one indexer, several nodes immediately go

down

with the 'Error adding log' message.

I'm using CloudSolrClient.add(List) to do the
indexing.  Is this correct for SolrCloud?

Thank you!

-Joe



On 7/17/2017 8:36 AM, Joe Obernberger wrote:


We've been indexing data on a 45 node cluster with 100 shards and 3
replicas, but our indexing processes have been stopping due to errors.
On
the server side the error is "Error logging add". Stack trace:

2017-07-17 12:29:24.057 INFO  (qtp985934102-5161548) [c:UNCLASS
s:shard58
r:core_node290 x:UNCLASS_shard58_replica1]
o.a.s.u.p.LogUpdateProcessorFactory
[UNCLASS_shard58_replica1] webapp=/solr path=/update
params={update.distrib=FROMLEADER=add-unknown-
fields-to-the-schema=http://tarvos:9100/solr/
UNCLASS_shard58_replica2/=javabin=2}{add=[
COLLECT20003218348784 (1573172872544780288), COLLECT20003218351447
(1573172872620277760), COLLECT20003218353085 (1573172872625520640),
COLLECT20003218357937 (1573172872627617792), COLLECT20003218361860
(1573172872629714944), COLLECT20003218362535 (1573172872631812096)]} 0
171
2017-07-17 12:29:24.160 INFO  (qtp985934102-5160762) [c:UNCLASS
s:shard13
r:core_node81 x:UNCLASS_shard13_replica1]
o.a.s.u.p.LogUpdateProcessorFactory
[UNCLASS_shard13_replica1] webapp=/solr path=/update
params={update.distrib=FROMLEADER=add-unknown-
fields-to-the-schema=http://tarvos:9100/solr/
UNCLASS_shard13_replica2/=javabin=2}{add=[
COLLECT20003218344436 (1573172872538488832), COLLECT20003218347497
(1573172872620277760), COLLECT20003218351645 (1573172872625520640),
COLLECT20003218356965 (1573172872629714944), COLLECT20003218357775
(1573172872632860672), COLLECT20003218358017 (1573172872646492160),
COLLECT20003218358152 (1573172872650686464), COLLECT20003218359395
(1573172872651735040), COLLECT20003218362571 (1573172872652783616)]} 0
274
2017-07-17 12:29:24.163 INFO  (qtp985934102-5161057) [c:UNCLASS
s:shard43
r:core_node108 

Re: How to exclude stop words in spellcheck collations

[Susheel Kumar]

The field which you are using for spellcheck suggestions can utilise
stopword filter factory.

Thanks,
Susheel

On Sun, Jul 16, 2017 at 12:47 PM, Naveen Pajjuri 
wrote:

> Hi,
> Is there any way i can exclude stop words from the collations and
> sugesstions from spell check component ?
>
> Regards,
> Naveen Pajjuri.
>

Re: Solr Subfaceting

[Amrit Sarkar]

Poornima,

  1.  In confluence - https://cwiki.apache.org/confluence/display/solr/
Faceted+Search it page says its experimental and may change significantly.
Is it safe for us to use the Terms faceting or will it change in future
releases?. When will this be official?.

A lot of people / engineers are using json faceting in their production
today itself. By "experimental and may change significantly" simple means
the end points of request and response may change in in future releases,
hence the back-compat will suffer. If you are upgrading to future released
solr version, you have to make sure the client code you have wrote at your
end (via SolrJ) is upto date with that solr version (you upgrade to).

  2.  As Term faceting has few advantages over Pivot facet as per
http://yonik.com/solr-subfacets/ we went on with it. Is it safe to use it
or do we use Pivot faceting instead?

In my opinion, you should use the better feature. Though you may hit some
limitations of json faceting and their respective would be jiras opened too.

Rest Mr. Seeley would be the the best person the 2nd.

Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2

On Mon, Jul 17, 2017 at 10:43 PM, Ponnuswamy, Poornima (GE Healthcare) <
poornima.ponnusw...@ge.com> wrote:

> Thanks for your response. I have tried with SolrParams and it works for me.
>
> Any feedback on question 1 & 2.
>
> Thanks,
> Poornima
>
> On 7/17/17, 12:38 PM, "Amrit Sarkar"  wrote:
>
> Poornima,
>
> Regarding 3;
> You can do something like:
>
> CloudSolrClient client = new CloudSolrClient("localhost:9983");
>
> SolrParams params = new ModifiableSolrParams().add("q","*:*")
> .add("json.facet","{.}");
>
> QueryResponse response = client.query(params);
>
> Setting key and value via SolrParams is available.
>
>
> Amrit Sarkar
> Search Engineer
> Lucidworks, Inc.
> 415-589-9269
> www.lucidworks.com
> Twitter http://twitter.com/lucidworks
> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>
> On Mon, Jul 17, 2017 at 8:48 PM, Ponnuswamy, Poornima (GE Healthcare) <
> poornima.ponnusw...@ge.com> wrote:
>
> > Hello,
> >
> > We have Solr version 6.4.2  and we have been using Solr Subfaceting –
> > Terms Facet as per the document https://cwiki.apache.org/
> > confluence/display/solr/Faceted+Search in our project.
> >
> > In our project which is going to go in production soon, we use it for
> > getting the facet/subfacet counts, sort etc. We make a direct rest
> call to
> > solr and the counts matches perfectly. I have few questions and
> > clarification on this approach and appreciate your response on this.
> >
> >
> >
> >   1.  In confluence - https://cwiki.apache.org/
> confluence/display/solr/
> > Faceted+Search it page says its experimental and may change
> > significantly. Is it safe for us to use the Terms faceting or will it
> > change in future releases?. When will this be official?.
> >   2.  As Term faceting has few advantages over Pivot facet as per
> > http://yonik.com/solr-subfacets/ we went on with it. Is it safe to
> use it
> > or do we use Pivot faceting instead?
> >   3.  Currently we make a rest call to Solr API to get results. Now
> we are
> > planning to move to Solr Cloud and use Solrj library to integrate
> with
> > Solr. I don’t see any support for Terms faceting (json.facet) in
> Solrj
> > library. Am I overlooking it or will it be supported in future
> releases?
> >
> > Appreciate your response.
> >
> > Thanks,
> > Poornima
> >
> >
>
>
>

Re: Solr Subfaceting

[Ponnuswamy, Poornima (GE Healthcare)]

Thanks for your response. I have tried with SolrParams and it works for me.

Any feedback on question 1 & 2. 

Thanks,
Poornima

On 7/17/17, 12:38 PM, "Amrit Sarkar"  wrote:

Poornima,

Regarding 3;
You can do something like:

CloudSolrClient client = new CloudSolrClient("localhost:9983");

SolrParams params = new ModifiableSolrParams().add("q","*:*")
.add("json.facet","{.}");

QueryResponse response = client.query(params);

Setting key and value via SolrParams is available.


Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2

On Mon, Jul 17, 2017 at 8:48 PM, Ponnuswamy, Poornima (GE Healthcare) <
poornima.ponnusw...@ge.com> wrote:

> Hello,
>
> We have Solr version 6.4.2  and we have been using Solr Subfaceting –
> Terms Facet as per the document https://cwiki.apache.org/
> confluence/display/solr/Faceted+Search in our project.
>
> In our project which is going to go in production soon, we use it for
> getting the facet/subfacet counts, sort etc. We make a direct rest call to
> solr and the counts matches perfectly. I have few questions and
> clarification on this approach and appreciate your response on this.
>
>
>
>   1.  In confluence - https://cwiki.apache.org/confluence/display/solr/
> Faceted+Search it page says its experimental and may change
> significantly. Is it safe for us to use the Terms faceting or will it
> change in future releases?. When will this be official?.
>   2.  As Term faceting has few advantages over Pivot facet as per
> http://yonik.com/solr-subfacets/ we went on with it. Is it safe to use it
> or do we use Pivot faceting instead?
>   3.  Currently we make a rest call to Solr API to get results. Now we are
> planning to move to Solr Cloud and use Solrj library to integrate with
> Solr. I don’t see any support for Terms faceting (json.facet) in Solrj
> library. Am I overlooking it or will it be supported in future releases?
>
> Appreciate your response.
>
> Thanks,
> Poornima
>
>

Re: Solr Subfaceting

[Amrit Sarkar]

Poornima,

Regarding 3;
You can do something like:

CloudSolrClient client = new CloudSolrClient("localhost:9983");

SolrParams params = new ModifiableSolrParams().add("q","*:*")
.add("json.facet","{.}");

QueryResponse response = client.query(params);

Setting key and value via SolrParams is available.


Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2

On Mon, Jul 17, 2017 at 8:48 PM, Ponnuswamy, Poornima (GE Healthcare) <
poornima.ponnusw...@ge.com> wrote:

> Hello,
>
> We have Solr version 6.4.2  and we have been using Solr Subfaceting –
> Terms Facet as per the document https://cwiki.apache.org/
> confluence/display/solr/Faceted+Search in our project.
>
> In our project which is going to go in production soon, we use it for
> getting the facet/subfacet counts, sort etc. We make a direct rest call to
> solr and the counts matches perfectly. I have few questions and
> clarification on this approach and appreciate your response on this.
>
>
>
>   1.  In confluence - https://cwiki.apache.org/confluence/display/solr/
> Faceted+Search it page says its experimental and may change
> significantly. Is it safe for us to use the Terms faceting or will it
> change in future releases?. When will this be official?.
>   2.  As Term faceting has few advantages over Pivot facet as per
> http://yonik.com/solr-subfacets/ we went on with it. Is it safe to use it
> or do we use Pivot faceting instead?
>   3.  Currently we make a rest call to Solr API to get results. Now we are
> planning to move to Solr Cloud and use Solrj library to integrate with
> Solr. I don’t see any support for Terms faceting (json.facet) in Solrj
> library. Am I overlooking it or will it be supported in future releases?
>
> Appreciate your response.
>
> Thanks,
> Poornima
>
>

Re: TransactionLog doesn't know how to serialize class java.util.UUID; try implementing ObjectResolver?

[Amrit Sarkar]

I looked into the code TransactionLog.java (branch_5_5) ::

JavaBinCodec.ObjectResolver resolver = new JavaBinCodec.ObjectResolver() {
  @Override
  public Object resolve(Object o, JavaBinCodec codec) throws IOException {
if (o instanceof BytesRef) {
  BytesRef br = (BytesRef)o;
  codec.writeByteArray(br.bytes, br.offset, br.length);
  return null;
}
// Fallback: we have no idea how to serialize this.  Be noisy to
prevent insidious bugs
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR,
"TransactionLog doesn't know how to serialize " + o.getClass()
+ "; try implementing ObjectResolver?");
  }
};

While UUID implements serializable, so should be BytesRef instance to?? ::

public final class UUID implements java.io.Serializable, Comparable

Can you share the payload with you are trying to update?



Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2

On Mon, Jul 17, 2017 at 7:03 PM, deviantcode  wrote:

> Hi Mahmoud, did you ever get to the bottom of this? I'm having the same
> issue
> on solr 5.5.2
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/TransactionLog-doesn-t-know-how-to-serialize-
> class-java-util-UUID-try-implementing-ObjectResolver-
> tp4332277p4346335.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Solr 6.6.0 - Indexing errors

[Joe Obernberger]

Erick - thank you.  I meant to disable field guessing as our indexer 
does this internally.  Thanks for seeing that!  Yes, we've seen things 
come in like IDs that are 12345 (int), but then next ID is 12AF456 (string).


There is also a version mismatch between our Cloudera 5.10.2 hadoop 
version and the version shipped with 6.6.0; correcting that.

Thanks again!

-Joe


On 7/17/2017 11:53 AM, Erick Erickson wrote:

Joe:

I agree that 46 million docs later you'd expect things to have settled
out. However, I do note that you have
"add-unknown-fields-to-the-schema" in your error stack which means
you're using "field guessing", sometimes called data_driven. I would
recommend you do _not_ use this for production as, while it does the
best job it can it has to make assumptions about what the data looks
like based on the first document it sees which may later be violated.
Getting "possible analysis error" is one of the messages that happens
when this occurs.

The simple example is that if the first time data_driven sees "1"
it'll guess integer. If sometime later there's a doc with "1.0" it'll
generate a parse error.

I totally agree that 46 million docs later you'd expect all of this
kind of thing to have flushed out, but the "possible analysis error"
seems to be pointing that direction. If this is, indeed, the problem
you'll see better evidence on the Solr instance that's actually having
the problem. Unfortunately you'll just to look at one Solr log from
each shard to see whether this is an issue.

Best,
Erick

On Mon, Jul 17, 2017 at 7:23 AM, Joe Obernberger
 wrote:

So far we've indexed about 46 million documents, but over the weekend, these
errors started coming up.  I would expect that if there was a basic issue,
it would have started right away?  We ran a test cluster with just a few
shards/replicas prior and didn't see any issues using the same indexing
code, but we're running a lot more indexers simultaneously with the larger
cluster; perhaps we're just overloading HDFS?  The same nodes that run Solr
also run HDFS datanodes, but they are pretty beefy machines; we're not
swapping.

As Shawn pointed out, I will be checking the HDFS version (we're using
Cloudera CDH 5.10.2), and the HDFS logs.

-Joe



On 7/17/2017 10:16 AM, Susheel Kumar wrote:

There is some analysis error also.  I would suggest to test the indexer on
just one shard setup first, then test for a replica (1 shard and 1
replica)
and then test for 2 shards and 2 replica.  This would confirm if there is
basic issue with indexing / cluster setup.

On Mon, Jul 17, 2017 at 9:04 AM, Joe Obernberger <
joseph.obernber...@gmail.com> wrote:


Some more info:

When I stop all the indexers, in about 5-10 minutes the cluster goes all
green.  When I start just one indexer, several nodes immediately go down
with the 'Error adding log' message.

I'm using CloudSolrClient.add(List) to do the
indexing.  Is this correct for SolrCloud?

Thank you!

-Joe



On 7/17/2017 8:36 AM, Joe Obernberger wrote:


We've been indexing data on a 45 node cluster with 100 shards and 3
replicas, but our indexing processes have been stopping due to errors.
On
the server side the error is "Error logging add". Stack trace:

2017-07-17 12:29:24.057 INFO  (qtp985934102-5161548) [c:UNCLASS
s:shard58
r:core_node290 x:UNCLASS_shard58_replica1]
o.a.s.u.p.LogUpdateProcessorFactory
[UNCLASS_shard58_replica1] webapp=/solr path=/update
params={update.distrib=FROMLEADER=add-unknown-
fields-to-the-schema=http://tarvos:9100/solr/
UNCLASS_shard58_replica2/=javabin=2}{add=[
COLLECT20003218348784 (1573172872544780288), COLLECT20003218351447
(1573172872620277760), COLLECT20003218353085 (1573172872625520640),
COLLECT20003218357937 (1573172872627617792), COLLECT20003218361860
(1573172872629714944), COLLECT20003218362535 (1573172872631812096)]} 0
171
2017-07-17 12:29:24.160 INFO  (qtp985934102-5160762) [c:UNCLASS
s:shard13
r:core_node81 x:UNCLASS_shard13_replica1]
o.a.s.u.p.LogUpdateProcessorFactory
[UNCLASS_shard13_replica1] webapp=/solr path=/update
params={update.distrib=FROMLEADER=add-unknown-
fields-to-the-schema=http://tarvos:9100/solr/
UNCLASS_shard13_replica2/=javabin=2}{add=[
COLLECT20003218344436 (1573172872538488832), COLLECT20003218347497
(1573172872620277760), COLLECT20003218351645 (1573172872625520640),
COLLECT20003218356965 (1573172872629714944), COLLECT20003218357775
(1573172872632860672), COLLECT20003218358017 (1573172872646492160),
COLLECT20003218358152 (1573172872650686464), COLLECT20003218359395
(1573172872651735040), COLLECT20003218362571 (1573172872652783616)]} 0
274
2017-07-17 12:29:24.163 INFO  (qtp985934102-5161057) [c:UNCLASS
s:shard43
r:core_node108 x:UNCLASS_shard43_replica1]
o.a.s.u.p.LogUpdateProcessorFactory
[UNCLASS_shard43_replica1] webapp=/solr path=/update
params={update.distrib=FROMLEADER=add-unknown-
fields-to-the-schema=http://tarvos:9100/solr/
UNCLASS_shard43_replica2/=javabin=2}{} 0 0
2017-07-17 12:29:24.164 ERROR 

Re: solr-user-subscribe

[srshaik]

I added a reply to the discussion. Please accept.

On Fri, Jul 14, 2017 at 11:05 PM, Naohiko Uramoto [via Lucene] <
ml+s472066n4346101...@n3.nabble.com> wrote:

> solr-user-subscribe <[hidden email]
> >
>
> --
> Naohiko Uramoto
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
> http://lucene.472066.n3.nabble.com/solr-user-subscribe-tp4346101.html
> To unsubscribe from Solr - User, click here
> 
> .
> NAML
> 
>




--
View this message in context: 
http://lucene.472066.n3.nabble.com/solr-user-subscribe-tp4346101p4346307.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr 6.6.0 - Indexing errors

[Susheel Kumar]

and there is document id mentioned above when it failed with analysis
error.  You can look how those documents differ as Eric suggested.

On Mon, Jul 17, 2017 at 11:53 AM, Erick Erickson 
wrote:

> Joe:
>
> I agree that 46 million docs later you'd expect things to have settled
> out. However, I do note that you have
> "add-unknown-fields-to-the-schema" in your error stack which means
> you're using "field guessing", sometimes called data_driven. I would
> recommend you do _not_ use this for production as, while it does the
> best job it can it has to make assumptions about what the data looks
> like based on the first document it sees which may later be violated.
> Getting "possible analysis error" is one of the messages that happens
> when this occurs.
>
> The simple example is that if the first time data_driven sees "1"
> it'll guess integer. If sometime later there's a doc with "1.0" it'll
> generate a parse error.
>
> I totally agree that 46 million docs later you'd expect all of this
> kind of thing to have flushed out, but the "possible analysis error"
> seems to be pointing that direction. If this is, indeed, the problem
> you'll see better evidence on the Solr instance that's actually having
> the problem. Unfortunately you'll just to look at one Solr log from
> each shard to see whether this is an issue.
>
> Best,
> Erick
>
> On Mon, Jul 17, 2017 at 7:23 AM, Joe Obernberger
>  wrote:
> > So far we've indexed about 46 million documents, but over the weekend,
> these
> > errors started coming up.  I would expect that if there was a basic
> issue,
> > it would have started right away?  We ran a test cluster with just a few
> > shards/replicas prior and didn't see any issues using the same indexing
> > code, but we're running a lot more indexers simultaneously with the
> larger
> > cluster; perhaps we're just overloading HDFS?  The same nodes that run
> Solr
> > also run HDFS datanodes, but they are pretty beefy machines; we're not
> > swapping.
> >
> > As Shawn pointed out, I will be checking the HDFS version (we're using
> > Cloudera CDH 5.10.2), and the HDFS logs.
> >
> > -Joe
> >
> >
> >
> > On 7/17/2017 10:16 AM, Susheel Kumar wrote:
> >>
> >> There is some analysis error also.  I would suggest to test the indexer
> on
> >> just one shard setup first, then test for a replica (1 shard and 1
> >> replica)
> >> and then test for 2 shards and 2 replica.  This would confirm if there
> is
> >> basic issue with indexing / cluster setup.
> >>
> >> On Mon, Jul 17, 2017 at 9:04 AM, Joe Obernberger <
> >> joseph.obernber...@gmail.com> wrote:
> >>
> >>> Some more info:
> >>>
> >>> When I stop all the indexers, in about 5-10 minutes the cluster goes
> all
> >>> green.  When I start just one indexer, several nodes immediately go
> down
> >>> with the 'Error adding log' message.
> >>>
> >>> I'm using CloudSolrClient.add(List) to do the
> >>> indexing.  Is this correct for SolrCloud?
> >>>
> >>> Thank you!
> >>>
> >>> -Joe
> >>>
> >>>
> >>>
> >>> On 7/17/2017 8:36 AM, Joe Obernberger wrote:
> >>>
>  We've been indexing data on a 45 node cluster with 100 shards and 3
>  replicas, but our indexing processes have been stopping due to errors.
>  On
>  the server side the error is "Error logging add". Stack trace:
> 
>  2017-07-17 12:29:24.057 INFO  (qtp985934102-5161548) [c:UNCLASS
>  s:shard58
>  r:core_node290 x:UNCLASS_shard58_replica1]
>  o.a.s.u.p.LogUpdateProcessorFactory
>  [UNCLASS_shard58_replica1] webapp=/solr path=/update
>  params={update.distrib=FROMLEADER=add-unknown-
>  fields-to-the-schema=http://tarvos:9100/solr/
>  UNCLASS_shard58_replica2/=javabin=2}{add=[
>  COLLECT20003218348784 (1573172872544780288), COLLECT20003218351447
>  (1573172872620277760), COLLECT20003218353085 (1573172872625520640),
>  COLLECT20003218357937 (1573172872627617792), COLLECT20003218361860
>  (1573172872629714944), COLLECT20003218362535 (1573172872631812096)]} 0
>  171
>  2017-07-17 12:29:24.160 INFO  (qtp985934102-5160762) [c:UNCLASS
>  s:shard13
>  r:core_node81 x:UNCLASS_shard13_replica1]
>  o.a.s.u.p.LogUpdateProcessorFactory
>  [UNCLASS_shard13_replica1] webapp=/solr path=/update
>  params={update.distrib=FROMLEADER=add-unknown-
>  fields-to-the-schema=http://tarvos:9100/solr/
>  UNCLASS_shard13_replica2/=javabin=2}{add=[
>  COLLECT20003218344436 (1573172872538488832), COLLECT20003218347497
>  (1573172872620277760), COLLECT20003218351645 (1573172872625520640),
>  COLLECT20003218356965 (1573172872629714944), COLLECT20003218357775
>  (1573172872632860672), COLLECT20003218358017 (1573172872646492160),
>  COLLECT20003218358152 (1573172872650686464), COLLECT20003218359395
>  (1573172872651735040), COLLECT20003218362571 (1573172872652783616)]} 0
>  274
>  2017-07-17 12:29:24.163 INFO  (qtp985934102-5161057) [c:UNCLASS
>  s:shard43
> 

Re: solr-user-subscribe

[Erick Erickson]

Please follow the instructions here:
http://lucene.apache.org/solr/community.html#mailing-lists-irc. You
must use the _exact_ same e-mail as you used to subscribe.


If the initial try doesn't work and following the suggestions at the
"problems" link doesn't work for you, let us know. But note you need
to show us the _entire_ return header to allow anyone to diagnose the
problem.


Best,

Erick

On Sun, Jul 16, 2017 at 12:49 PM, Yangrui Guo  wrote:
> unsubscribe
>
> On Friday, July 14, 2017, Naohiko Uramoto  wrote:
>
>> solr-user-subscribe >
>>
>> --
>> Naohiko Uramoto
>>

Re: Get results in multiple orders (multiple boosts)

[Erick Erickson]

I don't think boosting is really what you want here. Boosting
_influences_ the score, it does not impose an ordering.

Sorting _does_ impose an ordering, the question is how to sort and the
answer depends on how fixed (or not) the sorting criteria are. Do they
change with different queries? If not, the very simplest thing to do
is to index a field with a pre-computed sort value. IOW, if your
ordering is _always_ source 5, 9, 7 index a source_sort field that
orders things that way and sort on that. Then I'd have a secondary
sort by score as a tie-breaker.

If that's not the case, perhaps sorting by function (perhaps a custom
function) would work.

Best,
Erick

On Mon, Jul 17, 2017 at 4:30 AM, Florian Waltersdorfer
 wrote:
> Hi,
>
> I am quite the SolR newbie myself, but have you looked at the resulting 
> scores, e.g. via fl=*,score (that way, you can see/test how your boosting 
> affects the results)?
> In a similar scenario, I am using fixed value boosts for specific field 
> values; "^=[boost]" instead of "^[factor]", for example:
>
> category:9500^=20  source:(5^=20 OR 9^=10 OR 7^=5)
>
> (Actual fixed values open for experimentation.)
>
> Regards,
> Florian
>
> -Ursprüngliche Nachricht-
> Von: Luca Dall'Osto [mailto:tenacious...@yahoo.it.INVALID]
> Gesendet: Montag, 17. Juli 2017 12:20
> An: solr-user@lucene.apache.org
> Betreff: Get results in multiple orders (multiple boosts)
>
> Hello,
> I'm new in Solr (and in mailing lists..), and I have a question about 
> querying contents in multiple custom orders.
> I 'm trying to query some documents boosted by 2 (or more) fields: I'm able 
> to make a search of 2 day and return results boosted by category field, like 
> this:
>
> ?indent=on
> =edismax
> =(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
> =category:9500^2
> =category:1100^1
> =40
> =jsonThis will return all documents of category 9500 first, and 1100 in 
> after.Now I would like to get this documents with a second boost based on 
> another field, called source.I would like to have documents in this order:1) 
> category:9500 AND source:5
> 2) category:9500 AND source:9
> 3) category:9500 AND source:7
> 4) category:1100 AND source:5
> 5) category:1100 AND source:9
> 6) category:1100 AND source:7
> To get this order, I tied with this query:?indent=on =edismax 
> =(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
> =category:9500^2+source:(5^3 OR 9^2 OR 7^1)
> =category:1100^1+source:(5^3 OR 9^2 OR 7^1)
> =40
> =json
> How can I apply a double boosts to get the documents in my correct order? Is 
> boost the correct tool for my purpose?Any help will be greatly appreciated. 
> Thanks Luca

Re: dynamic datasource password in db_data_config file

[Amrit Sarkar]

Javed,

Can you let us know if you are running in standalone or cloud mode?

Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2

On Mon, Jul 17, 2017 at 11:54 AM, javeed  wrote:

> HI Team,
> Can you please update on this issue.
>
> Thank you
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/dynamic-datasource-password-in-db-data-config-
> file-tp4345804p4346288.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Solr 6.6.0 - Indexing errors

[Erick Erickson]

Joe:

I agree that 46 million docs later you'd expect things to have settled
out. However, I do note that you have
"add-unknown-fields-to-the-schema" in your error stack which means
you're using "field guessing", sometimes called data_driven. I would
recommend you do _not_ use this for production as, while it does the
best job it can it has to make assumptions about what the data looks
like based on the first document it sees which may later be violated.
Getting "possible analysis error" is one of the messages that happens
when this occurs.

The simple example is that if the first time data_driven sees "1"
it'll guess integer. If sometime later there's a doc with "1.0" it'll
generate a parse error.

I totally agree that 46 million docs later you'd expect all of this
kind of thing to have flushed out, but the "possible analysis error"
seems to be pointing that direction. If this is, indeed, the problem
you'll see better evidence on the Solr instance that's actually having
the problem. Unfortunately you'll just to look at one Solr log from
each shard to see whether this is an issue.

Best,
Erick

On Mon, Jul 17, 2017 at 7:23 AM, Joe Obernberger
 wrote:
> So far we've indexed about 46 million documents, but over the weekend, these
> errors started coming up.  I would expect that if there was a basic issue,
> it would have started right away?  We ran a test cluster with just a few
> shards/replicas prior and didn't see any issues using the same indexing
> code, but we're running a lot more indexers simultaneously with the larger
> cluster; perhaps we're just overloading HDFS?  The same nodes that run Solr
> also run HDFS datanodes, but they are pretty beefy machines; we're not
> swapping.
>
> As Shawn pointed out, I will be checking the HDFS version (we're using
> Cloudera CDH 5.10.2), and the HDFS logs.
>
> -Joe
>
>
>
> On 7/17/2017 10:16 AM, Susheel Kumar wrote:
>>
>> There is some analysis error also.  I would suggest to test the indexer on
>> just one shard setup first, then test for a replica (1 shard and 1
>> replica)
>> and then test for 2 shards and 2 replica.  This would confirm if there is
>> basic issue with indexing / cluster setup.
>>
>> On Mon, Jul 17, 2017 at 9:04 AM, Joe Obernberger <
>> joseph.obernber...@gmail.com> wrote:
>>
>>> Some more info:
>>>
>>> When I stop all the indexers, in about 5-10 minutes the cluster goes all
>>> green.  When I start just one indexer, several nodes immediately go down
>>> with the 'Error adding log' message.
>>>
>>> I'm using CloudSolrClient.add(List) to do the
>>> indexing.  Is this correct for SolrCloud?
>>>
>>> Thank you!
>>>
>>> -Joe
>>>
>>>
>>>
>>> On 7/17/2017 8:36 AM, Joe Obernberger wrote:
>>>
 We've been indexing data on a 45 node cluster with 100 shards and 3
 replicas, but our indexing processes have been stopping due to errors.
 On
 the server side the error is "Error logging add". Stack trace:

 2017-07-17 12:29:24.057 INFO  (qtp985934102-5161548) [c:UNCLASS
 s:shard58
 r:core_node290 x:UNCLASS_shard58_replica1]
 o.a.s.u.p.LogUpdateProcessorFactory
 [UNCLASS_shard58_replica1] webapp=/solr path=/update
 params={update.distrib=FROMLEADER=add-unknown-
 fields-to-the-schema=http://tarvos:9100/solr/
 UNCLASS_shard58_replica2/=javabin=2}{add=[
 COLLECT20003218348784 (1573172872544780288), COLLECT20003218351447
 (1573172872620277760), COLLECT20003218353085 (1573172872625520640),
 COLLECT20003218357937 (1573172872627617792), COLLECT20003218361860
 (1573172872629714944), COLLECT20003218362535 (1573172872631812096)]} 0
 171
 2017-07-17 12:29:24.160 INFO  (qtp985934102-5160762) [c:UNCLASS
 s:shard13
 r:core_node81 x:UNCLASS_shard13_replica1]
 o.a.s.u.p.LogUpdateProcessorFactory
 [UNCLASS_shard13_replica1] webapp=/solr path=/update
 params={update.distrib=FROMLEADER=add-unknown-
 fields-to-the-schema=http://tarvos:9100/solr/
 UNCLASS_shard13_replica2/=javabin=2}{add=[
 COLLECT20003218344436 (1573172872538488832), COLLECT20003218347497
 (1573172872620277760), COLLECT20003218351645 (1573172872625520640),
 COLLECT20003218356965 (1573172872629714944), COLLECT20003218357775
 (1573172872632860672), COLLECT20003218358017 (1573172872646492160),
 COLLECT20003218358152 (1573172872650686464), COLLECT20003218359395
 (1573172872651735040), COLLECT20003218362571 (1573172872652783616)]} 0
 274
 2017-07-17 12:29:24.163 INFO  (qtp985934102-5161057) [c:UNCLASS
 s:shard43
 r:core_node108 x:UNCLASS_shard43_replica1]
 o.a.s.u.p.LogUpdateProcessorFactory
 [UNCLASS_shard43_replica1] webapp=/solr path=/update
 params={update.distrib=FROMLEADER=add-unknown-
 fields-to-the-schema=http://tarvos:9100/solr/
 UNCLASS_shard43_replica2/=javabin=2}{} 0 0
 2017-07-17 12:29:24.164 ERROR (qtp985934102-5161057) [c:UNCLASS
 s:shard43
 r:core_node108 x:UNCLASS_shard43_replica1] 

Re: Create too many zookeeper connections when recreate CloudSolrServer instance

[Walter Underwood]

If your Zookeeper cluster is rebooting frequently, you have much, much worse 
problems than client connections.

Is Zookeeper unstable in your installation? If so, fix that.

Stop hacking the client. 

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)


> On Jul 17, 2017, at 1:48 AM, wg85907  wrote:
> 
> Hi Shawn,
>Thanks for your detail explanation. The reason I want to shutdown
> the CloudSolrServer instance and create a new one is that I have concern
> that if it can successfully reconnect to Zookeeper server if Zookeeper
> cluster has some issue and reboot. I will do related test with version
> 6.5.0, which is the version I want to upgrade to. If there is any issue, I
> will report the issue to you and your team as you suggested. Anyway I will
> abandon the way that shutdown/close the CloudSolrServer instance and create
> a new one. The alternative opinion is to manage Zookeeper connection myself
> by extending Class ZkClientClusterStateProvider. 
> Regards,
> Geng, Wei
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Create-too-many-zookeeper-connections-when-recreate-CloudSolrServer-instance-tp4346040p4346295.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: dynamic datasource password in db_data_config file

[javeed]

HI Team,
Can you please update on this issue.

Thank you




--
View this message in context: 
http://lucene.472066.n3.nabble.com/dynamic-datasource-password-in-db-data-config-file-tp4345804p4346288.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Creating a custom auth plugin for solr

[srshaik]

Thanks Jan. I had gone though the link. But, not the code. I will look into
it and try to understand. However, I had a question regarding support
multi-tenancy. 
If I have one collection containing documents for multiple tenants, would I
have to build a custom authorization plugin to prevent one customer from
updating the document of another. Or should I build a custom handler for
each of the operations such as update / delete / search etc..



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Creating-a-custom-auth-plugin-for-solr-tp4345762p4346301.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Creating a custom auth plugin for solr

[srshaik]

Thanks Jan. I had gone though the link. But, not the code. I will look into
it and try to understand. However, I had a question regarding support
multi-tenancy. 
If I have one collection containing documents for multiple tenants, would I
have to build a custom authorization plugin to prevent one customer from
updating the document of another. Or should I build a custom handler for
each of the operations such as update / delete / search etc..



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Creating-a-custom-auth-plugin-for-solr-tp4345762p4346358.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: TransactionLog doesn't know how to serialize class java.util.UUID; try implementing ObjectResolver?

[deviantcode]

Hi Mahmoud, did you ever get to the bottom of this? I'm having the same issue
on solr 5.5.2



--
View this message in context: 
http://lucene.472066.n3.nabble.com/TransactionLog-doesn-t-know-how-to-serialize-class-java-util-UUID-try-implementing-ObjectResolver-tp4332277p4346335.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr Subfaceting

[Ponnuswamy, Poornima (GE Healthcare)]

Hello,

We have Solr version 6.4.2  and we have been using Solr Subfaceting – Terms 
Facet as per the document 
https://cwiki.apache.org/confluence/display/solr/Faceted+Search in our project.

In our project which is going to go in production soon, we use it for getting 
the facet/subfacet counts, sort etc. We make a direct rest call to solr and the 
counts matches perfectly. I have few questions and clarification on this 
approach and appreciate your response on this.



  1.  In confluence - 
https://cwiki.apache.org/confluence/display/solr/Faceted+Search it page says 
its experimental and may change significantly. Is it safe for us to use the 
Terms faceting or will it change in future releases?. When will this be 
official?.
  2.  As Term faceting has few advantages over Pivot facet as per 
http://yonik.com/solr-subfacets/ we went on with it. Is it safe to use it or do 
we use Pivot faceting instead?
  3.  Currently we make a rest call to Solr API to get results. Now we are 
planning to move to Solr Cloud and use Solrj library to integrate with Solr. I 
don’t see any support for Terms faceting (json.facet) in Solrj library. Am I 
overlooking it or will it be supported in future releases?

Appreciate your response.

Thanks,
Poornima

Re: Solr 6.6.0 - Indexing errors

[Joe Obernberger]

So far we've indexed about 46 million documents, but over the weekend, 
these errors started coming up.  I would expect that if there was a 
basic issue, it would have started right away?  We ran a test cluster 
with just a few shards/replicas prior and didn't see any issues using 
the same indexing code, but we're running a lot more indexers 
simultaneously with the larger cluster; perhaps we're just overloading 
HDFS?  The same nodes that run Solr also run HDFS datanodes, but they 
are pretty beefy machines; we're not swapping.


As Shawn pointed out, I will be checking the HDFS version (we're using 
Cloudera CDH 5.10.2), and the HDFS logs.


-Joe


On 7/17/2017 10:16 AM, Susheel Kumar wrote:

There is some analysis error also.  I would suggest to test the indexer on
just one shard setup first, then test for a replica (1 shard and 1 replica)
and then test for 2 shards and 2 replica.  This would confirm if there is
basic issue with indexing / cluster setup.

On Mon, Jul 17, 2017 at 9:04 AM, Joe Obernberger <
joseph.obernber...@gmail.com> wrote:


Some more info:

When I stop all the indexers, in about 5-10 minutes the cluster goes all
green.  When I start just one indexer, several nodes immediately go down
with the 'Error adding log' message.

I'm using CloudSolrClient.add(List) to do the
indexing.  Is this correct for SolrCloud?

Thank you!

-Joe



On 7/17/2017 8:36 AM, Joe Obernberger wrote:


We've been indexing data on a 45 node cluster with 100 shards and 3
replicas, but our indexing processes have been stopping due to errors.  On
the server side the error is "Error logging add". Stack trace:

2017-07-17 12:29:24.057 INFO  (qtp985934102-5161548) [c:UNCLASS s:shard58
r:core_node290 x:UNCLASS_shard58_replica1] o.a.s.u.p.LogUpdateProcessorFactory
[UNCLASS_shard58_replica1] webapp=/solr path=/update
params={update.distrib=FROMLEADER=add-unknown-
fields-to-the-schema=http://tarvos:9100/solr/
UNCLASS_shard58_replica2/=javabin=2}{add=[
COLLECT20003218348784 (1573172872544780288), COLLECT20003218351447
(1573172872620277760), COLLECT20003218353085 (1573172872625520640),
COLLECT20003218357937 (1573172872627617792), COLLECT20003218361860
(1573172872629714944), COLLECT20003218362535 (1573172872631812096)]} 0 171
2017-07-17 12:29:24.160 INFO  (qtp985934102-5160762) [c:UNCLASS s:shard13
r:core_node81 x:UNCLASS_shard13_replica1] o.a.s.u.p.LogUpdateProcessorFactory
[UNCLASS_shard13_replica1] webapp=/solr path=/update
params={update.distrib=FROMLEADER=add-unknown-
fields-to-the-schema=http://tarvos:9100/solr/
UNCLASS_shard13_replica2/=javabin=2}{add=[
COLLECT20003218344436 (1573172872538488832), COLLECT20003218347497
(1573172872620277760), COLLECT20003218351645 (1573172872625520640),
COLLECT20003218356965 (1573172872629714944), COLLECT20003218357775
(1573172872632860672), COLLECT20003218358017 (1573172872646492160),
COLLECT20003218358152 (1573172872650686464), COLLECT20003218359395
(1573172872651735040), COLLECT20003218362571 (1573172872652783616)]} 0 274
2017-07-17 12:29:24.163 INFO  (qtp985934102-5161057) [c:UNCLASS s:shard43
r:core_node108 x:UNCLASS_shard43_replica1] o.a.s.u.p.LogUpdateProcessorFactory
[UNCLASS_shard43_replica1] webapp=/solr path=/update
params={update.distrib=FROMLEADER=add-unknown-
fields-to-the-schema=http://tarvos:9100/solr/
UNCLASS_shard43_replica2/=javabin=2}{} 0 0
2017-07-17 12:29:24.164 ERROR (qtp985934102-5161057) [c:UNCLASS s:shard43
r:core_node108 x:UNCLASS_shard43_replica1] o.a.s.h.RequestHandlerBase
org.apache.solr.common.SolrException: Error logging add
 at org.apache.solr.update.TransactionLog.write(TransactionLog.
java:418)
 at org.apache.solr.update.UpdateLog.add(UpdateLog.java:532)
 at org.apache.solr.update.UpdateLog.add(UpdateLog.java:516)
 at org.apache.solr.update.processor.DistributedUpdateProcessor.
versionAdd(DistributedUpdateProcessor.java:1113)
 at org.apache.solr.update.processor.DistributedUpdateProcessor.
processAdd(DistributedUpdateProcessor.java:748)
 at org.apache.solr.update.processor.LogUpdateProcessorFactory$L
ogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
 at org.apache.solr.handler.loader.JavabinLoader$1.update(Javabi
nLoader.java:98)
 at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCod
ec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:180)
 at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCod
ec$1.readIterator(JavaBinUpdateRequestCodec.java:136)
 at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinC
odec.java:306)
 at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCode
c.java:251)
 at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCod
ec$1.readNamedList(JavaBinUpdateRequestCodec.java:122)
 at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinC
odec.java:271)
 at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCode
c.java:251)
 at 

Re: Solr 6.6.0 - Indexing errors

[Susheel Kumar]

There is some analysis error also.  I would suggest to test the indexer on
just one shard setup first, then test for a replica (1 shard and 1 replica)
and then test for 2 shards and 2 replica.  This would confirm if there is
basic issue with indexing / cluster setup.

On Mon, Jul 17, 2017 at 9:04 AM, Joe Obernberger <
joseph.obernber...@gmail.com> wrote:

> Some more info:
>
> When I stop all the indexers, in about 5-10 minutes the cluster goes all
> green.  When I start just one indexer, several nodes immediately go down
> with the 'Error adding log' message.
>
> I'm using CloudSolrClient.add(List) to do the
> indexing.  Is this correct for SolrCloud?
>
> Thank you!
>
> -Joe
>
>
>
> On 7/17/2017 8:36 AM, Joe Obernberger wrote:
>
>> We've been indexing data on a 45 node cluster with 100 shards and 3
>> replicas, but our indexing processes have been stopping due to errors.  On
>> the server side the error is "Error logging add". Stack trace:
>>
>> 2017-07-17 12:29:24.057 INFO  (qtp985934102-5161548) [c:UNCLASS s:shard58
>> r:core_node290 x:UNCLASS_shard58_replica1] 
>> o.a.s.u.p.LogUpdateProcessorFactory
>> [UNCLASS_shard58_replica1] webapp=/solr path=/update
>> params={update.distrib=FROMLEADER=add-unknown-
>> fields-to-the-schema=http://tarvos:9100/solr/
>> UNCLASS_shard58_replica2/=javabin=2}{add=[
>> COLLECT20003218348784 (1573172872544780288), COLLECT20003218351447
>> (1573172872620277760), COLLECT20003218353085 (1573172872625520640),
>> COLLECT20003218357937 (1573172872627617792), COLLECT20003218361860
>> (1573172872629714944), COLLECT20003218362535 (1573172872631812096)]} 0 171
>> 2017-07-17 12:29:24.160 INFO  (qtp985934102-5160762) [c:UNCLASS s:shard13
>> r:core_node81 x:UNCLASS_shard13_replica1] o.a.s.u.p.LogUpdateProcessorFactory
>> [UNCLASS_shard13_replica1] webapp=/solr path=/update
>> params={update.distrib=FROMLEADER=add-unknown-
>> fields-to-the-schema=http://tarvos:9100/solr/
>> UNCLASS_shard13_replica2/=javabin=2}{add=[
>> COLLECT20003218344436 (1573172872538488832), COLLECT20003218347497
>> (1573172872620277760), COLLECT20003218351645 (1573172872625520640),
>> COLLECT20003218356965 (1573172872629714944), COLLECT20003218357775
>> (1573172872632860672), COLLECT20003218358017 (1573172872646492160),
>> COLLECT20003218358152 (1573172872650686464), COLLECT20003218359395
>> (1573172872651735040), COLLECT20003218362571 (1573172872652783616)]} 0 274
>> 2017-07-17 12:29:24.163 INFO  (qtp985934102-5161057) [c:UNCLASS s:shard43
>> r:core_node108 x:UNCLASS_shard43_replica1] 
>> o.a.s.u.p.LogUpdateProcessorFactory
>> [UNCLASS_shard43_replica1] webapp=/solr path=/update
>> params={update.distrib=FROMLEADER=add-unknown-
>> fields-to-the-schema=http://tarvos:9100/solr/
>> UNCLASS_shard43_replica2/=javabin=2}{} 0 0
>> 2017-07-17 12:29:24.164 ERROR (qtp985934102-5161057) [c:UNCLASS s:shard43
>> r:core_node108 x:UNCLASS_shard43_replica1] o.a.s.h.RequestHandlerBase
>> org.apache.solr.common.SolrException: Error logging add
>> at org.apache.solr.update.TransactionLog.write(TransactionLog.
>> java:418)
>> at org.apache.solr.update.UpdateLog.add(UpdateLog.java:532)
>> at org.apache.solr.update.UpdateLog.add(UpdateLog.java:516)
>> at org.apache.solr.update.processor.DistributedUpdateProcessor.
>> versionAdd(DistributedUpdateProcessor.java:1113)
>> at org.apache.solr.update.processor.DistributedUpdateProcessor.
>> processAdd(DistributedUpdateProcessor.java:748)
>> at org.apache.solr.update.processor.LogUpdateProcessorFactory$L
>> ogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
>> at org.apache.solr.handler.loader.JavabinLoader$1.update(Javabi
>> nLoader.java:98)
>> at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCod
>> ec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:180)
>> at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCod
>> ec$1.readIterator(JavaBinUpdateRequestCodec.java:136)
>> at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinC
>> odec.java:306)
>> at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCode
>> c.java:251)
>> at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCod
>> ec$1.readNamedList(JavaBinUpdateRequestCodec.java:122)
>> at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinC
>> odec.java:271)
>> at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCode
>> c.java:251)
>> at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCo
>> dec.java:173)
>> at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCod
>> ec.unmarshal(JavaBinUpdateRequestCodec.java:187)
>> at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDoc
>> s(JavabinLoader.java:108)
>> at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoa
>> der.java:55)
>> at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRe
>> questHandler.java:97)
>> at 

Re: Solr 6.6.0 - Indexing errors

[Shawn Heisey]

On 7/17/2017 6:36 AM, Joe Obernberger wrote:
> We've been indexing data on a 45 node cluster with 100 shards and 3
> replicas, but our indexing processes have been stopping due to
> errors.  On the server side the error is "Error logging add". Stack
> trace:
 
> Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException):
> File
> /solr6.6.0/UNCLASS/core_node108/data/tlog/tlog.0006211
> could only be replicated to 0 nodes instead of minReplication (=1). 
> There are 40 datanode(s) running and no node(s) are excluded in this
> operation.

The excerpt from your log that I preserved above shows that the root of
the problem is something going wrong with Solr writing to HDFS.  I can
only tell that there was a problem, I do not what actually went wrong.

I think you'll need to take this information to the hadoop project and
ask them what could cause it and what can be done about it.

Solr includes Hadoop 2.7.2 jars.  This is not the latest version of
Hadoop, so it's possible there might be a known issue with this version
that is fixed in a later version.  There is a task to update Solr's
Hadoop to 3.0 when it gets released:

https://issues.apache.org/jira/browse/SOLR-9515

Thanks,
Shawn

Re: Cant stop/start server

[Susheel Kumar]

Exactly.  Both are different and for the purpose if you see the content.
The later refers the prev one.

On Mon, Jul 17, 2017 at 9:15 AM, Iridian Group 
wrote:

> So I installed SOLR on another server using just the service install
> script and am experiencing the same issue when starting/stopping the
> service using /opt/solr/bin/solr stop -all
>
> however when using
> /etc/init.d/solr start
> /etc/init.d/solr stop
> the server starts/stops gracefully without issue.
>
> Running as a service must tweak the install in a way that makes manual
> start/stop not operate or is invalid in the context of a service install.
>
> K
>
>
>
> > On Jul 14, 2017, at 7:58 PM, Erick Erickson 
> wrote:
> >
> > Hmm, looks like the pid file is located in different spots (depending)
> > and the -p option apparently looks in the same place but the -all
> > doesn't. Haven't tracked down why
> >
> > If I start by
> >
> > bin/solr start -s example/techproducts/configs
> > the pid file goes in bin/solr
> > It also goes there if I cd into the bin directory and:
> > ./solr start -s
> > /Users/Erick/apache/solrJiras/jira/solr/example/techproducts/solr/
> >
> >
> > However, if I cd into the bin directory then:
> > ./solr start -s ../example/techproducts/solr
> > the pid file goes in to ../example/techproducts/solr
> > when the pid file goes here, the -all doesn't find it.
> >
> >
> > The odd thing is that it's findable in all cases by the -p option but
> > not the -all option.
> >
> > Seems like a problem with the script, but I'll leave it to someone
> > else. In the mean time what happens when you start with an absolute
> > path? Or at least without a ../ as the start of your path?
> >
> > Erick
> >
> >
> > On Fri, Jul 14, 2017 at 11:12 AM, Iridian Group
> >  wrote:
> >> REL 7.3
> >> Apache 2.4.6
> >>
> >> Sry, not versed enough in CLI to get your ‘find’ to work. Dropped me
> into a prompt of some type. Got this however.
> >> find / -name "solr-*.pid"
> >> /var/solr/solr-8983.pid
> >>
> >>
> >>
> >> Join us on facebook  or twitter
> 
> >>> On Jul 14, 2017, at 12:56 PM, Erick Erickson 
> wrote:
> >>>
> >>> Shouldn't be a setup or configuration issue, it should "just happen".
> >>> But if this has been up and running for a long time perhaps someone
> >>> "cleaned it up".
> >>>
> >>> Hmmm, now that I think about it the pid file must have been there if
> >>> "-p " worked so I'm stumped too. What op system? The relevant
> >>> part of the *nix script is:
> >>>
> >>> find "$SOLR_PID_DIR" -name "solr-*.pid" -type f | while read PIDF
> >>>
> >>> and windows is:
> >>> set found_it=0
> >>> for /f "usebackq" %%i in (`dir /b "%SOLR_TIP%\bin" ^| findstr /i
> >>> "^solr-.*\.port$"`) do (
> >>>   set SOME_SOLR_PORT=
> >>>
> >>> Just wonder if they're depending on something not in your system?
> >>>
> >>> Best,
> >>> Erick
> >>>
> >>> On Fri, Jul 14, 2017 at 10:26 AM, Iridian Group
> >>> > wrote:
>  Typical story, I wasn’t the admin who set it up but I’m pretty sure
> is was vanilla.
> 
> 
>  Thanks
> 
>  Keith Savoie
>  Vice President of Technology
> 
>  IRiDiAN GROUP
> 
>  Helping organizations brand
>  & market themselves through
>  web, print, & social media.
> 
> 
>  14450 Eagle Run Dr. Ste. 120
>  Omaha, Nebraska 68116
> 
>  P  • 402.422.0150
>  W • iridiangroup.com  <
> https://www.iridiangroup.com/ >
> 
>  Join us on facebook  https://www.facebook.com/iridiangroup>> or twitter  iridiangroup >
> > On Jul 14, 2017, at 12:18 PM, Erick Erickson <
> erickerick...@gmail.com> wrote:
> >
> > bq: wonder why -all didn’t pick it up?
> >
> > Good question, I use this _all_ the time. (little joke there).
> >
> > The -all flag looks for various .pid files, you'll see things like:
> > solr-8983.pid that contain the process id to kill associated with
> that
> > port. Any chance these were removed or in some different place?
> >
> > Erick
> >
> > On Fri, Jul 14, 2017 at 10:15 AM, Iridian Group
> >  wrote:
> >> Ahhh well then.
> >> I did try the -all flag but it returned nothing.
> >>
> >> However an explicit  -p 8983 did the trick.  :)
> >>
> >> … wonder why -all didn’t pick it up?
> >>
> >> Thanks!
> >>
> >>
> >>
> >> Keith Savoie
> >> Vice President of Technology
> >>
> >> IRiDiAN GROUP
> >>
> >> Helping organizations brand
> >> & market themselves through
> >> web, print, & social media.
> >>
> >>
> >> 14450 

Re: CDCR - how to deal with the transaction log files

[Susheel Kumar]

I just voted for https://issues.apache.org/jira/browse/SOLR-11069 to get it
resolved, as we are discussing to start using CDCR soon.

On Fri, Jul 14, 2017 at 5:21 PM, Varun Thacker  wrote:

> https://issues.apache.org/jira/browse/SOLR-11069 is tracking why is
> LASTPROCESSEDVERSION=-1
> on the source cluster always
>
> On Fri, Jul 14, 2017 at 11:46 AM, jmyatt  wrote:
>
> > Thanks for the suggestion - tried that today and still no luck.  Time to
> > write a script to naively / blindly delete old logs and run that in cron.
> > *sigh*
> >
> >
> >
> > --
> > View this message in context: http://lucene.472066.n3.
> > nabble.com/CDCR-how-to-deal-with-the-transaction-log-
> > files-tp4345062p4346138.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
>

Re: Cant stop/start server

[Iridian Group]

So I installed SOLR on another server using just the service install script and 
am experiencing the same issue when starting/stopping the service using 
/opt/solr/bin/solr stop -all

however when using
/etc/init.d/solr start
/etc/init.d/solr stop
the server starts/stops gracefully without issue. 

Running as a service must tweak the install in a way that makes manual 
start/stop not operate or is invalid in the context of a service install. 

K



> On Jul 14, 2017, at 7:58 PM, Erick Erickson  wrote:
> 
> Hmm, looks like the pid file is located in different spots (depending)
> and the -p option apparently looks in the same place but the -all
> doesn't. Haven't tracked down why
> 
> If I start by
> 
> bin/solr start -s example/techproducts/configs
> the pid file goes in bin/solr
> It also goes there if I cd into the bin directory and:
> ./solr start -s
> /Users/Erick/apache/solrJiras/jira/solr/example/techproducts/solr/
> 
> 
> However, if I cd into the bin directory then:
> ./solr start -s ../example/techproducts/solr
> the pid file goes in to ../example/techproducts/solr
> when the pid file goes here, the -all doesn't find it.
> 
> 
> The odd thing is that it's findable in all cases by the -p option but
> not the -all option.
> 
> Seems like a problem with the script, but I'll leave it to someone
> else. In the mean time what happens when you start with an absolute
> path? Or at least without a ../ as the start of your path?
> 
> Erick
> 
> 
> On Fri, Jul 14, 2017 at 11:12 AM, Iridian Group
>  wrote:
>> REL 7.3
>> Apache 2.4.6
>> 
>> Sry, not versed enough in CLI to get your ‘find’ to work. Dropped me into a 
>> prompt of some type. Got this however.
>> find / -name "solr-*.pid"
>> /var/solr/solr-8983.pid
>> 
>> 
>> 
>> Join us on facebook  or twitter 
>> 
>>> On Jul 14, 2017, at 12:56 PM, Erick Erickson  
>>> wrote:
>>> 
>>> Shouldn't be a setup or configuration issue, it should "just happen".
>>> But if this has been up and running for a long time perhaps someone
>>> "cleaned it up".
>>> 
>>> Hmmm, now that I think about it the pid file must have been there if
>>> "-p " worked so I'm stumped too. What op system? The relevant
>>> part of the *nix script is:
>>> 
>>> find "$SOLR_PID_DIR" -name "solr-*.pid" -type f | while read PIDF
>>> 
>>> and windows is:
>>> set found_it=0
>>> for /f "usebackq" %%i in (`dir /b "%SOLR_TIP%\bin" ^| findstr /i
>>> "^solr-.*\.port$"`) do (
>>>   set SOME_SOLR_PORT=
>>> 
>>> Just wonder if they're depending on something not in your system?
>>> 
>>> Best,
>>> Erick
>>> 
>>> On Fri, Jul 14, 2017 at 10:26 AM, Iridian Group
>>> > wrote:
 Typical story, I wasn’t the admin who set it up but I’m pretty sure is was 
 vanilla.
 
 
 Thanks
 
 Keith Savoie
 Vice President of Technology
 
 IRiDiAN GROUP
 
 Helping organizations brand
 & market themselves through
 web, print, & social media.
 
 
 14450 Eagle Run Dr. Ste. 120
 Omaha, Nebraska 68116
 
 P  • 402.422.0150
 W • iridiangroup.com  
 >
 
 Join us on facebook > or twitter 
 >
> On Jul 14, 2017, at 12:18 PM, Erick Erickson  
> wrote:
> 
> bq: wonder why -all didn’t pick it up?
> 
> Good question, I use this _all_ the time. (little joke there).
> 
> The -all flag looks for various .pid files, you'll see things like:
> solr-8983.pid that contain the process id to kill associated with that
> port. Any chance these were removed or in some different place?
> 
> Erick
> 
> On Fri, Jul 14, 2017 at 10:15 AM, Iridian Group
>  wrote:
>> Ahhh well then.
>> I did try the -all flag but it returned nothing.
>> 
>> However an explicit  -p 8983 did the trick.  :)
>> 
>> … wonder why -all didn’t pick it up?
>> 
>> Thanks!
>> 
>> 
>> 
>> Keith Savoie
>> Vice President of Technology
>> 
>> IRiDiAN GROUP
>> 
>> Helping organizations brand
>> & market themselves through
>> web, print, & social media.
>> 
>> 
>> 14450 Eagle Run Dr. Ste. 120
>> Omaha, Nebraska 68116
>> 
>> P  • 402.422.0150
>> W • iridiangroup.com 
>> 
>> Join us on facebook  or twitter 
>> 
>>> On Jul 14, 2017, at 12:08 PM, Atita Arora  wrote:
>>> 
>>> Did you mention the port 

Re: Solr 6.6.0 - Indexing errors

[Joe Obernberger]

Some more info:

When I stop all the indexers, in about 5-10 minutes the cluster goes all 
green.  When I start just one indexer, several nodes immediately go down 
with the 'Error adding log' message.


I'm using CloudSolrClient.add(List) to do the 
indexing.  Is this correct for SolrCloud?


Thank you!

-Joe


On 7/17/2017 8:36 AM, Joe Obernberger wrote:
We've been indexing data on a 45 node cluster with 100 shards and 3 
replicas, but our indexing processes have been stopping due to 
errors.  On the server side the error is "Error logging add". Stack 
trace:


2017-07-17 12:29:24.057 INFO  (qtp985934102-5161548) [c:UNCLASS 
s:shard58 r:core_node290 x:UNCLASS_shard58_replica1] 
o.a.s.u.p.LogUpdateProcessorFactory [UNCLASS_shard58_replica1] 
webapp=/solr path=/update 
params={update.distrib=FROMLEADER=add-unknown-fields-to-the-schema=http://tarvos:9100/solr/UNCLASS_shard58_replica2/=javabin=2}{add=[COLLECT20003218348784 
(1573172872544780288), COLLECT20003218351447 (1573172872620277760), 
COLLECT20003218353085 (1573172872625520640), COLLECT20003218357937 
(1573172872627617792), COLLECT20003218361860 (1573172872629714944), 
COLLECT20003218362535 (1573172872631812096)]} 0 171
2017-07-17 12:29:24.160 INFO  (qtp985934102-5160762) [c:UNCLASS 
s:shard13 r:core_node81 x:UNCLASS_shard13_replica1] 
o.a.s.u.p.LogUpdateProcessorFactory [UNCLASS_shard13_replica1] 
webapp=/solr path=/update 
params={update.distrib=FROMLEADER=add-unknown-fields-to-the-schema=http://tarvos:9100/solr/UNCLASS_shard13_replica2/=javabin=2}{add=[COLLECT20003218344436 
(1573172872538488832), COLLECT20003218347497 (1573172872620277760), 
COLLECT20003218351645 (1573172872625520640), COLLECT20003218356965 
(1573172872629714944), COLLECT20003218357775 (1573172872632860672), 
COLLECT20003218358017 (1573172872646492160), COLLECT20003218358152 
(1573172872650686464), COLLECT20003218359395 (1573172872651735040), 
COLLECT20003218362571 (1573172872652783616)]} 0 274
2017-07-17 12:29:24.163 INFO  (qtp985934102-5161057) [c:UNCLASS 
s:shard43 r:core_node108 x:UNCLASS_shard43_replica1] 
o.a.s.u.p.LogUpdateProcessorFactory [UNCLASS_shard43_replica1] 
webapp=/solr path=/update 
params={update.distrib=FROMLEADER=add-unknown-fields-to-the-schema=http://tarvos:9100/solr/UNCLASS_shard43_replica2/=javabin=2}{} 
0 0
2017-07-17 12:29:24.164 ERROR (qtp985934102-5161057) [c:UNCLASS 
s:shard43 r:core_node108 x:UNCLASS_shard43_replica1] 
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error 
logging add
at 
org.apache.solr.update.TransactionLog.write(TransactionLog.java:418)

at org.apache.solr.update.UpdateLog.add(UpdateLog.java:532)
at org.apache.solr.update.UpdateLog.add(UpdateLog.java:516)
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1113)
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:748)
at 
org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
at 
org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:98)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:180)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:136)
at 
org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:306)
at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:251)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:122)
at 
org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:271)
at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:251)
at 
org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:173)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:187)
at 
org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:108)
at 
org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55)
at 
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
at 
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at 

Solr 6.6.0 - Indexing errors

[Joe Obernberger]

We've been indexing data on a 45 node cluster with 100 shards and 3 
replicas, but our indexing processes have been stopping due to errors.  
On the server side the error is "Error logging add". Stack trace:


2017-07-17 12:29:24.057 INFO  (qtp985934102-5161548) [c:UNCLASS 
s:shard58 r:core_node290 x:UNCLASS_shard58_replica1] 
o.a.s.u.p.LogUpdateProcessorFactory [UNCLASS_shard58_replica1] 
webapp=/solr path=/update 
params={update.distrib=FROMLEADER=add-unknown-fields-to-the-schema=http://tarvos:9100/solr/UNCLASS_shard58_replica2/=javabin=2}{add=[COLLECT20003218348784 
(1573172872544780288), COLLECT20003218351447 (1573172872620277760), 
COLLECT20003218353085 (1573172872625520640), COLLECT20003218357937 
(1573172872627617792), COLLECT20003218361860 (1573172872629714944), 
COLLECT20003218362535 (1573172872631812096)]} 0 171
2017-07-17 12:29:24.160 INFO  (qtp985934102-5160762) [c:UNCLASS 
s:shard13 r:core_node81 x:UNCLASS_shard13_replica1] 
o.a.s.u.p.LogUpdateProcessorFactory [UNCLASS_shard13_replica1] 
webapp=/solr path=/update 
params={update.distrib=FROMLEADER=add-unknown-fields-to-the-schema=http://tarvos:9100/solr/UNCLASS_shard13_replica2/=javabin=2}{add=[COLLECT20003218344436 
(1573172872538488832), COLLECT20003218347497 (1573172872620277760), 
COLLECT20003218351645 (1573172872625520640), COLLECT20003218356965 
(1573172872629714944), COLLECT20003218357775 (1573172872632860672), 
COLLECT20003218358017 (1573172872646492160), COLLECT20003218358152 
(1573172872650686464), COLLECT20003218359395 (1573172872651735040), 
COLLECT20003218362571 (1573172872652783616)]} 0 274
2017-07-17 12:29:24.163 INFO  (qtp985934102-5161057) [c:UNCLASS 
s:shard43 r:core_node108 x:UNCLASS_shard43_replica1] 
o.a.s.u.p.LogUpdateProcessorFactory [UNCLASS_shard43_replica1] 
webapp=/solr path=/update 
params={update.distrib=FROMLEADER=add-unknown-fields-to-the-schema=http://tarvos:9100/solr/UNCLASS_shard43_replica2/=javabin=2}{} 
0 0
2017-07-17 12:29:24.164 ERROR (qtp985934102-5161057) [c:UNCLASS 
s:shard43 r:core_node108 x:UNCLASS_shard43_replica1] 
o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error 
logging add
at 
org.apache.solr.update.TransactionLog.write(TransactionLog.java:418)

at org.apache.solr.update.UpdateLog.add(UpdateLog.java:532)
at org.apache.solr.update.UpdateLog.add(UpdateLog.java:516)
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1113)
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:748)
at 
org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
at 
org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:98)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:180)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:136)
at 
org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:306)
at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:251)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:122)
at 
org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:271)
at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:251)
at 
org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:173)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:187)
at 
org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:108)
at 
org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55)
at 
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
at 
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)

at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 

AW: Get results in multiple orders (multiple boosts)

[Florian Waltersdorfer]

Hi,

I am quite the SolR newbie myself, but have you looked at the resulting scores, 
e.g. via fl=*,score (that way, you can see/test how your boosting affects the 
results)?
In a similar scenario, I am using fixed value boosts for specific field values; 
"^=[boost]" instead of "^[factor]", for example:

category:9500^=20  source:(5^=20 OR 9^=10 OR 7^=5)

(Actual fixed values open for experimentation.)

Regards,
Florian

-Ursprüngliche Nachricht-
Von: Luca Dall'Osto [mailto:tenacious...@yahoo.it.INVALID] 
Gesendet: Montag, 17. Juli 2017 12:20
An: solr-user@lucene.apache.org
Betreff: Get results in multiple orders (multiple boosts)

Hello,
I'm new in Solr (and in mailing lists..), and I have a question about querying 
contents in multiple custom orders.
I 'm trying to query some documents boosted by 2 (or more) fields: I'm able to 
make a search of 2 day and return results boosted by category field, like this:

?indent=on
=edismax
=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
=category:9500^2
=category:1100^1
=40
=jsonThis will return all documents of category 9500 first, and 1100 in 
after.Now I would like to get this documents with a second boost based on 
another field, called source.I would like to have documents in this order:1) 
category:9500 AND source:5
2) category:9500 AND source:9
3) category:9500 AND source:7
4) category:1100 AND source:5
5) category:1100 AND source:9
6) category:1100 AND source:7
To get this order, I tied with this query:?indent=on =edismax 
=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z])
=category:9500^2+source:(5^3 OR 9^2 OR 7^1)
=category:1100^1+source:(5^3 OR 9^2 OR 7^1)
=40
=json
How can I apply a double boosts to get the documents in my correct order? Is 
boost the correct tool for my purpose?Any help will be greatly appreciated. 
Thanks Luca

Get results in multiple orders (multiple boosts)

[Luca Dall'Osto]

Hello,
I'm new in Solr (and in mailing lists..), and I have a question about querying 
contents in multiple custom orders.
I 'm trying to query some documents boosted by 2 (or more) fields: I'm able to 
make a search of 2 day and return results boosted by category field, like this:

?indent=on
=edismax
=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z]) 
=category:9500^2
=category:1100^1
=40
=jsonThis will return all documents of category 9500 first, and 1100 in 
after.Now I would like to get this documents with a second boost based on 
another field, called source.I would like to have documents in this order:1) 
category:9500 AND source:5
2) category:9500 AND source:9
3) category:9500 AND source:7
4) category:1100 AND source:5
5) category:1100 AND source:9
6) category:1100 AND source:7
To get this order, I tied with this query:?indent=on
=edismax
=(date:[2017-06-16T00:00:00Z TO 2017-06-18T23:59:59Z]) 
=category:9500^2+source:(5^3 OR 9^2 OR 7^1)
=category:1100^1+source:(5^3 OR 9^2 OR 7^1)
=40
=json
How can I apply a double boosts to get the documents in my correct order? Is 
boost the correct tool for my purpose?Any help will be greatly appreciated. 
Thanks
Luca

Re: Create too many zookeeper connections when recreate CloudSolrServer instance

[wg85907]

Hi Shawn,
Thanks for your detail explanation. The reason I want to shutdown
the CloudSolrServer instance and create a new one is that I have concern
that if it can successfully reconnect to Zookeeper server if Zookeeper
cluster has some issue and reboot. I will do related test with version
6.5.0, which is the version I want to upgrade to. If there is any issue, I
will report the issue to you and your team as you suggested. Anyway I will
abandon the way that shutdown/close the CloudSolrServer instance and create
a new one. The alternative opinion is to manage Zookeeper connection myself
by extending Class ZkClientClusterStateProvider. 
Regards,
Geng, Wei



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Create-too-many-zookeeper-connections-when-recreate-CloudSolrServer-instance-tp4346040p4346295.html
Sent from the Solr - User mailing list archive at Nabble.com.

Parent child documents partial update

[Sujay Bawaskar]

Hi,

Need a help to understand solr parent child document partial update
behaviour. Can we perform partial update on parent document without losing
its chiild documents? My observation is that parent child relationship
between documents get lost in case partial update is performed on parent.
Any work around or solution to this issue?

-- 
Thanks,
Sujay P Bawaskar
M:+91-77091 53669