Re: Block Join Update

[Lucky Sharma]

Sure Mikhail,
One question on this,  If we wish to do this by creating a custom
update processor for handling this, do we also need to check for race
conditions?
On Mon, Dec 3, 2018 at 8:21 PM Mikhail Khludnev  wrote:
>
> So far, wipe whole block insert one as whole.
>
> On Mon, Dec 3, 2018 at 5:13 PM Lucky Sharma  wrote:
>
> > Hi,
> > Thanks, Mikhail.
> > But what about the suggestions for v6.6+?
> >
> > Regards,
> > Lucky Sharma
> > On Mon, Dec 3, 2018 at 7:07 PM Mikhail Khludnev  wrote:
> > >
> > > Hi,
> > > This might be improved in 8.0
> > > see https://issues.apache.org/jira/browse/SOLR-5211
> > >
> > > On Mon, Dec 3, 2018 at 2:39 PM Lucky Sharma  wrote:
> > >
> > > > Hi,
> > > > I have a query regarding block join update,
> > > > As far as I know, we cannot update the single doc of a block, we have
> > > > to delete the complete block and reindex it again.
> > > > Please clarify, If there is something wrong in my understanding.
> > > > So for the update in either parent or child, is it recommended to
> > > > write our own UpdateProcessor to handle block updates. if we don't
> > > > want to send the complete block in the request.
> > > >
> > > > If yes what happens during a race condition.
> > > >
> > > >
> > > > --
> > > > Warm Regards,
> > > >
> > > > Lucky Sharma
> > > > Contact No :+91 9821559918
> > > >
> > >
> > >
> > > --
> > > Sincerely yours
> > > Mikhail Khludnev
> >
> >
> >
> > --
> > Warm Regards,
> >
> > Lucky Sharma
> > Contact No :+91 9821559918
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev



-- 
Warm Regards,

Lucky Sharma
Contact No :+91 9821559918

Re: solrj - Batching and Optimistic Concurrency

[Erick Erickson]

You can add, say, a ScriptUpdateProcessor that checks this for you
pretty easily.

Have you looked at the Overwrite=false option (assuming you're not
assigning _version_ yourself)?

Best,
Erick
On Mon, Dec 3, 2018 at 11:57 AM lstusr 5u93n4  wrote:
>
> Hi All,
>
> I have a scenario where I'm trying to enable batching on the solrj client,
> but trying to see how that works with Optimistic Concurrency.
>
> From what I can tell, if I pass a list of SolrInputDocument to my solr
> client, and a document somewhere in that list contains a `_version_` field
> that would cause the Optimistic Concurrency check to fail:
>  - all documents in the list before the conflicting doc get saved correctly.
>  - no documents in the list after the conflicting doc get saved.
>
> What I would really like is to "send a list of documents to solr, set the
> _version_ on all of these documents to -1 so that they don't save if they
> already exist, and have solr save all of the "new" documents in the list".
>
> So three questions related to this:
>
> 1) Is Optimistic Concurrency the best mechanism for this, or is there some
> other "don't overwrite" flag I can set that would work better?
>
> 2) If Optimisic Concurrency is the right way to go, Is there a mode that I
> can set that would allow ALL non-conflicting documents in a batch to be
> saved?
>
> 3) If questions 1 or 2 are not possible, I could  trap the resulting
> RouteException  with a 409 code and remove the offending document from the
> list. But:
>   a) can I safely remove ALL documents in the list before the offending
> one, assuming they've been saved?
>   b) is there a better way to get the ID of the offending document besides
> parsing the 'Error from server at
> http://my.solr.instance:8983/solr/test_shard1_replica_n1: version conflict
> for doc2` string from the exception?
>
> Thanks!
>
> Kyle

Re: solrj - Batching and Optimistic Concurrency

[Erick Erickson]

And I forgot to mention TolerantUpdateProcessor, might be another approach.

On Mon, Dec 3, 2018 at 12:57 PM Erick Erickson  wrote:
>
> You can add, say, a ScriptUpdateProcessor that checks this for you
> pretty easily.
>
> Have you looked at the Overwrite=false option (assuming you're not
> assigning _version_ yourself)?
>
> Best,
> Erick
> On Mon, Dec 3, 2018 at 11:57 AM lstusr 5u93n4  wrote:
> >
> > Hi All,
> >
> > I have a scenario where I'm trying to enable batching on the solrj client,
> > but trying to see how that works with Optimistic Concurrency.
> >
> > From what I can tell, if I pass a list of SolrInputDocument to my solr
> > client, and a document somewhere in that list contains a `_version_` field
> > that would cause the Optimistic Concurrency check to fail:
> >  - all documents in the list before the conflicting doc get saved correctly.
> >  - no documents in the list after the conflicting doc get saved.
> >
> > What I would really like is to "send a list of documents to solr, set the
> > _version_ on all of these documents to -1 so that they don't save if they
> > already exist, and have solr save all of the "new" documents in the list".
> >
> > So three questions related to this:
> >
> > 1) Is Optimistic Concurrency the best mechanism for this, or is there some
> > other "don't overwrite" flag I can set that would work better?
> >
> > 2) If Optimisic Concurrency is the right way to go, Is there a mode that I
> > can set that would allow ALL non-conflicting documents in a batch to be
> > saved?
> >
> > 3) If questions 1 or 2 are not possible, I could  trap the resulting
> > RouteException  with a 409 code and remove the offending document from the
> > list. But:
> >   a) can I safely remove ALL documents in the list before the offending
> > one, assuming they've been saved?
> >   b) is there a better way to get the ID of the offending document besides
> > parsing the 'Error from server at
> > http://my.solr.instance:8983/solr/test_shard1_replica_n1: version conflict
> > for doc2` string from the exception?
> >
> > Thanks!
> >
> > Kyle

Re: Recover data from network shared path

[Erick Erickson]

Safest is to create a new collection, then shut it down. Now copy all the
indexes to the index sir for the corresponding replicas and start Solr back
up.

On Mon, Dec 3, 2018, 11:12 Rekha  Hi to all,My PC is physically damaged, so my Solr cloud instance is
> down.My Solr cloud instance data path is a network shared drive. So that
> all the indexed data are available. How to userecover that data by
> using the other  alternate Solr cloud instance. Thanks, Rekha
> Karthick

solrj - Batching and Optimistic Concurrency

[lstusr 5u93n4]

Hi All,

I have a scenario where I'm trying to enable batching on the solrj client,
but trying to see how that works with Optimistic Concurrency.

>From what I can tell, if I pass a list of SolrInputDocument to my solr
client, and a document somewhere in that list contains a `_version_` field
that would cause the Optimistic Concurrency check to fail:
 - all documents in the list before the conflicting doc get saved correctly.
 - no documents in the list after the conflicting doc get saved.

What I would really like is to "send a list of documents to solr, set the
_version_ on all of these documents to -1 so that they don't save if they
already exist, and have solr save all of the "new" documents in the list".

So three questions related to this:

1) Is Optimistic Concurrency the best mechanism for this, or is there some
other "don't overwrite" flag I can set that would work better?

2) If Optimisic Concurrency is the right way to go, Is there a mode that I
can set that would allow ALL non-conflicting documents in a batch to be
saved?

3) If questions 1 or 2 are not possible, I could  trap the resulting
RouteException  with a 409 code and remove the offending document from the
list. But:
  a) can I safely remove ALL documents in the list before the offending
one, assuming they've been saved?
  b) is there a better way to get the ID of the offending document besides
parsing the 'Error from server at
http://my.solr.instance:8983/solr/test_shard1_replica_n1: version conflict
for doc2` string from the exception?

Thanks!

Kyle

Recover data from network shared path

[Rekha]

Hi to all,My PC is physically damaged, so my Solr cloud instance is down.My 
Solr cloud instance data path is a network shared drive. So that all the 
indexed data are available. How to userecover that data by using the 
other  alternate Solr cloud instance. Thanks, Rekha Karthick 

RE: Re: Enable SSL for the existing SOLR Cloud Cluster

[Tech Support]

Hi Solr Team,

 

When I try to Implement the SSL for the Multi-Node SOLR Cloud Cluster.

 

Below in my SOLR CLOUD Cluster Environment,

 

* 3 Zookeeper PC running on the  version 3.4.13 - On Windows OS 

 

* 2 Solr Cloud Nodes running on the version 7.5.0  - on Windows OS.
Each Nodes running on individual PC.

 

In each SOLR I had generated the Self-Signed Certificate and used it for
each SOLR Instance.

 

I had already asked the same question
   and I already follow
the instructions, which is available in the SOLR Tutorial Link
  

 

I had faced the following problems, Please check and give the solution for
me.

 

1.   After enable SSL, I can able to access the SOLR Admin GUI with
HTTPS protocol. But, at the time of creating new collection by using the
Admin GUI, It gives the error message

Connection to Solr Lost, Please check the Solr Instance.

After restart all the Solr instance, Shards are created under one SOLR Node
only (Which I used to generate the Collection).  Shard Available on the
other nodes are shown in the down status.

 

2.   When I create the Collection from the command line. It gives the
error message like the below. But for this scenario, no Shard are created in
all the nodes.  

Failed to create collection  'mycollection' due to
:{SOLR-SERVER1-IP:8984_solr=org.apache.solr.client.solrj.SolrException: IO
Exception occurred when talking to server at : https://
SOLR-SERVER1-IP:8984/solr}

 

Please suggest me, in a right way.

 

Node: I need to implement SSL for already available Collection also. But,
but because of facing the above issue, I had used the fresh PC only (Without
existing Collections)

 

 

Thanks,

Karthick Ramu

 

 

From: Tech Support [mailto:techsupp...@sardonyx.in] 
Sent: Friday, November 30, 2018 7:23 PM
To: 'solr-user@lucene.apache.org'
Cc: 'edwinye...@gmail.com'
Subject: Re: Enable SSL for the existing SOLR Cloud Cluster

 

@  edwinye...@gmail.com  

Hi SOLR Team,
 
Thanks for your reply Zheng Lin Edwin Yeo mailto:edwinye...@gmail.com> >. 
 
I had already follow the same link and tried to install. After enable the
SSL, ADMIN GUI Open with HTTPS Only ( SSL implementation).
 
But unable to Create the New Collection.
 
At the time of trying for the new collection Admin GUI shows the warning
Alert. "Connection to Solr Lost, Please check the Solr Instance"
 
 
I am using 3 separate PC for Zookeeper, and 2 separate PC for SOLR instance.
 
Note : For all the installation, I am using Windows OS. For the Fresh
installation also I am getting this ERROR.
 

 

Thanks,

Karthick Ramu

 

 

 

Re: SolrCloud required ports

[Erick Erickson]

Others from the bin/solr script. Note that some are optional (JMX).
But to emphasize what Jan said: All these are configurable so you need
to make sure that whoever set up your system doesn't set these to
something else.

echo "  -p  Specify the port to start the Solr HTTP
listener on; default is 8983"
echo "  The specified port (SOLR_PORT) will also
be used to determine the stop port"
echo "  STOP_PORT=(\$SOLR_PORT-1000) and JMX RMI
listen port RMI_PORT=(\$SOLR_PORT+1). "
echo "  For instance, if you set -p 8985, then the
STOP_PORT=7985 and RMI_PORT=18985"
On Mon, Dec 3, 2018 at 4:00 AM Moshe Recanati | KMS  wrote:
>
> Hi Jan,
> Thank you.
> To summarize we need to open these ports within the cluster:
> 8983
> 2181
> 2888
> 3888
>
>
>
> Regards,
> Moshe Recanati
> CTO
> Mobile  + 972-52-6194481
> Skype:  recanati
>
> More at:  www.kmslh.com | LinkedIn | FB
>
> -Original Message-
> From: Jan Høydahl 
> Sent: Monday, December 3, 2018 12:43 PM
> To: solr-user 
> Subject: Re: SolrCloud required ports
>
> Hi
>
> This depends on your exact coniguration, so you should ask the engineers who 
> deployed ZK and Solr, not this list.
>
> If default solr port is used, you'd need at least 8983 open between servers 
> and from the app server to the cluster.
> If default zk port is used, you'd need port 2181 open between all three 
> servers but not externally (unless you use a client that needs to talk to zk) 
> Also zk needs to communicate internally in the quorum on two other ports, 
> which could be using ports 2888 and 3888 but could also be something else 
> depending on your exact configs. These will never need to be open outside the 
> cluster.
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> > 3. des. 2018 kl. 09:22 skrev Moshe Recanati | KMS :
> >
> > Hi,
> > We're currently running SolrCloud with 3 servers: 3 ZK and 3 Search 
> > Engines. Each one on each machine.
> > Our security team would like to open only the required ports between the 
> > servers.
> > Please let me know which ports we need to open between the servers?
> >
> > Thank you
> >
> > Regards,
> > Moshe Recanati
> > CTO
> > Mobile  + 972-52-6194481 
> > Skype:  recanati
> >
> >  > _news_kms-2Dlighthouse-2Dnamed-2Dgartner-2Dcool-2D121000184.html=DwI
> > FaQ=EtlJpXAqSaq3cSC4ACVw6-ifVo6KHbawEuqEp-kfN24=vNaquGtywQ6F1lNXYN
> > 9CVw=v6BhW17PQhjSm3ktWyaGQdUkg8AU6Dl5qw-QKMMoTFQ=uMsj8Lg5_yk5C70SF
> > Xpo5k1wRp5x-n55rw3x5L4iVwQ=> More at:  www.kmslh.com
> >  | LinkedIn
> >  > company_kms-2Dlighthouse=DwIFaQ=EtlJpXAqSaq3cSC4ACVw6-ifVo6KHbawEu
> > qEp-kfN24=vNaquGtywQ6F1lNXYN9CVw=v6BhW17PQhjSm3ktWyaGQdUkg8AU6Dl5q
> > w-QKMMoTFQ=WCIz8QUF02gSOunmgRennfMTdqBj6llOG0WkXzBurzc=> | FB
> >  > _pages_KMS-2Dlighthouse_123774257810917=DwIFaQ=EtlJpXAqSaq3cSC4ACV
> > w6-ifVo6KHbawEuqEp-kfN24=vNaquGtywQ6F1lNXYN9CVw=v6BhW17PQhjSm3ktWy
> > aGQdUkg8AU6Dl5qw-QKMMoTFQ=QI58JVs9eO7ARCUmSaJ4LVmBnR1unoV0jRSMBFhx7x
> > U=>

Re: Can I use configsets with custom stopwords per collection?

[Alexandre Rafalovitch]

Actually, just to correct myself. Solr uses configset in two different
ways (very unfortunate):
1) When you do bin/solr create -c name -d configset, in which case the
content of configset directory is copied
2) When you actually link to a configset as a common configuration, in
which case I think nothing is copied and core.properties points back
to the shared configset. In this case, any modifications affect all
the cores using the same backing configset.

I am guessing in the second interpretation, your question makes more
sense. I would then look at substitution variables and or managed stop
words, but I do not know the expected behavior.

Regards,
   Alex.
On Mon, 3 Dec 2018 at 11:05, Alexandre Rafalovitch  wrote:
>
> I am not sure I fully understand what you are saying.
>
> When you create a collection based on a configset, all the files
> should be copied, including the stopwords.
>
> You can also provide an absolute path.
>
> Solr also supports variable substitutions (as seen in solrconfig.xml
> library statements), but I am not actually sure if that applies to
> managed-schema. Could be an interesting test.
>
> Finally, you could use Managed configuration and push whatever set you
> need to whatever collection, but that's an extra step to be managed
> externally.
>
> Regards,
>Alex.
> On Mon, 3 Dec 2018 at 10:51, O. Klein  wrote:
> >
> > Yeah, but if i define them in the schema of configset, The custom file with
> > stopwords is in a directory relative to the collection and not in configset.
> >
> > So is there a way to define a path to stopwords with the collection as a
> > variable?
> >
> >
> >
> > --
> > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Can I use configsets with custom stopwords per collection?

[Alexandre Rafalovitch]

I am not sure I fully understand what you are saying.

When you create a collection based on a configset, all the files
should be copied, including the stopwords.

You can also provide an absolute path.

Solr also supports variable substitutions (as seen in solrconfig.xml
library statements), but I am not actually sure if that applies to
managed-schema. Could be an interesting test.

Finally, you could use Managed configuration and push whatever set you
need to whatever collection, but that's an extra step to be managed
externally.

Regards,
   Alex.
On Mon, 3 Dec 2018 at 10:51, O. Klein  wrote:
>
> Yeah, but if i define them in the schema of configset, The custom file with
> stopwords is in a directory relative to the collection and not in configset.
>
> So is there a way to define a path to stopwords with the collection as a
> variable?
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Can I use configsets with custom stopwords per collection?

[O. Klein]

Yeah, but if i define them in the schema of configset, The custom file with
stopwords is in a directory relative to the collection and not in configset.

So is there a way to define a path to stopwords with the collection as a
variable?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Can I use configsets with custom stopwords per collection?

[Alexandre Rafalovitch]

The stopwords are defined at the field type level as part of the
analyzer chain. So, you have per-field granularity. Not just
per-collection.

As stop-words are using files (though we have managed version as well,
you can share or not-share as much as you want even across different
field type definitions.

https://lucene.apache.org/solr/guide/7_5/filter-descriptions.html#stop-filter

Regards,
   Alex.
On Mon, 3 Dec 2018 at 10:04, O. Klein  wrote:
>
> Im using Solr standalone and I want to use shared stopwords and custom
> stopwords per collection. Is this possible?
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Can I use configsets with custom stopwords per collection?

[O. Klein]

Im using Solr standalone and I want to use shared stopwords and custom
stopwords per collection. Is this possible?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Block Join Update

[Mikhail Khludnev]

So far, wipe whole block insert one as whole.

On Mon, Dec 3, 2018 at 5:13 PM Lucky Sharma  wrote:

> Hi,
> Thanks, Mikhail.
> But what about the suggestions for v6.6+?
>
> Regards,
> Lucky Sharma
> On Mon, Dec 3, 2018 at 7:07 PM Mikhail Khludnev  wrote:
> >
> > Hi,
> > This might be improved in 8.0
> > see https://issues.apache.org/jira/browse/SOLR-5211
> >
> > On Mon, Dec 3, 2018 at 2:39 PM Lucky Sharma  wrote:
> >
> > > Hi,
> > > I have a query regarding block join update,
> > > As far as I know, we cannot update the single doc of a block, we have
> > > to delete the complete block and reindex it again.
> > > Please clarify, If there is something wrong in my understanding.
> > > So for the update in either parent or child, is it recommended to
> > > write our own UpdateProcessor to handle block updates. if we don't
> > > want to send the complete block in the request.
> > >
> > > If yes what happens during a race condition.
> > >
> > >
> > > --
> > > Warm Regards,
> > >
> > > Lucky Sharma
> > > Contact No :+91 9821559918
> > >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
>
>
>
> --
> Warm Regards,
>
> Lucky Sharma
> Contact No :+91 9821559918
>


-- 
Sincerely yours
Mikhail Khludnev

Re: Block Join Update

[Lucky Sharma]

Hi,
Thanks, Mikhail.
But what about the suggestions for v6.6+?

Regards,
Lucky Sharma
On Mon, Dec 3, 2018 at 7:07 PM Mikhail Khludnev  wrote:
>
> Hi,
> This might be improved in 8.0
> see https://issues.apache.org/jira/browse/SOLR-5211
>
> On Mon, Dec 3, 2018 at 2:39 PM Lucky Sharma  wrote:
>
> > Hi,
> > I have a query regarding block join update,
> > As far as I know, we cannot update the single doc of a block, we have
> > to delete the complete block and reindex it again.
> > Please clarify, If there is something wrong in my understanding.
> > So for the update in either parent or child, is it recommended to
> > write our own UpdateProcessor to handle block updates. if we don't
> > want to send the complete block in the request.
> >
> > If yes what happens during a race condition.
> >
> >
> > --
> > Warm Regards,
> >
> > Lucky Sharma
> > Contact No :+91 9821559918
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev



-- 
Warm Regards,

Lucky Sharma
Contact No :+91 9821559918

Re: Solr Request Handler

[Lucky Sharma]

Hi Alex & Andrea for the reply.
But Alex, Our main idea was to reduce network latency, since the only
processing needed is only i/p to the next call, which is totally Solr
params, like facets, sorting query etc. Thats the  reason I am looking
for the same.
Thanks, Andrea but in my case the cores are different.
Ex.
/reqHandler1 => core1
/reqHandler2 => core2


On Mon, Dec 3, 2018 at 6:01 PM Andrea Gazzarini  wrote:
>
> Hi,
> What Alexander said is right, but if in your scenario you would still go
> for that, you could try this [1], that should fit your need.
>
> Best,
> Andrea
>
> [1]  https://github.com/SeaseLtd/composite-request-handler
>
> On Mon, 3 Dec 2018, 13:26 Alexandre Rafalovitch 
> > You should not be exposing Solr directly to the client, but treating
> > it more as a database.
> >
> > Given that, why would you not write your processing code in that
> > middle-ware layer?
> >
> > Regards,
> >Alex.
> > On Mon, 3 Dec 2018 at 06:43, Lucky Sharma  wrote:
> > >
> > > Hi have one scenario,
> > > where I need to make a sequential call to the solr. Both the requests
> > > are sequential and output of the first will be required to set some
> > > params of the second search request.
> > >
> > > So for such scenarios, I am writing a plugin which will internally
> > > handle both the requests and gives the final response as output.
> > >
> > > Is this the correct approach?
> > > The only assumption I am making here is that both the cores are
> > > locally available.
> > >
> > > --
> > > Warm Regards,
> > >
> > > Lucky Sharma
> > > Contact No: +91 9821559918
> >



-- 
Warm Regards,

Lucky Sharma
Contact No :+91 9821559918

search devroom @ FOSDEM 2019

[Diego Ceccarelli (BLOOMBERG/ LONDON)]

Hi all, 

I just noticed this and I just wanted to share with you: 

Full-text search is everywhere nowadays and FOSDEM 2019 will have a dedicated 
devroom for search on Sunday the 3rd of February.

We would like to invite submissions of presentations from developers, 
researchers, and users of open source and free search software. Talk length 
should be one of 25 or 50 minutes (including questions).

The submission deadline is December 14th with notifications going out by 
Christmas.

Possible topics include:
* What is the state of full-text search in relational databases such as 
PostgreSQL or MySQL?
* How are search libraries like Apache Lucene evolving?
* What are current improvements and issues in popular search engines such as 
Elasticsearch and Solr?
* What is the role of machine learning in search and how do you even measure 
search quality?
* Which new systems are entering the search market?
* What is the place of client-side search tools like Fuse.js, Js Search, Lunr, 
and Elasticlunr.js?
* Are there any new break throughs in relevancy or are TF/IDF and BM25 good 
(enough)?

Please feel free to forward this announcement to any relevant FOSS project.

Submissions should be sent through https://penta.fosdem.org/submission/FOSDEM19.

https://lists.fosdem.org/pipermail/fosdem/2018q4/002805.html

Re: Block Join Update

[Mikhail Khludnev]

Hi,
This might be improved in 8.0
see https://issues.apache.org/jira/browse/SOLR-5211

On Mon, Dec 3, 2018 at 2:39 PM Lucky Sharma  wrote:

> Hi,
> I have a query regarding block join update,
> As far as I know, we cannot update the single doc of a block, we have
> to delete the complete block and reindex it again.
> Please clarify, If there is something wrong in my understanding.
> So for the update in either parent or child, is it recommended to
> write our own UpdateProcessor to handle block updates. if we don't
> want to send the complete block in the request.
>
> If yes what happens during a race condition.
>
>
> --
> Warm Regards,
>
> Lucky Sharma
> Contact No :+91 9821559918
>


-- 
Sincerely yours
Mikhail Khludnev

Re: Solr Request Handler

[Andrea Gazzarini]

Hi,
What Alexander said is right, but if in your scenario you would still go
for that, you could try this [1], that should fit your need.

Best,
Andrea

[1]  https://github.com/SeaseLtd/composite-request-handler

On Mon, 3 Dec 2018, 13:26 Alexandre Rafalovitch  You should not be exposing Solr directly to the client, but treating
> it more as a database.
>
> Given that, why would you not write your processing code in that
> middle-ware layer?
>
> Regards,
>Alex.
> On Mon, 3 Dec 2018 at 06:43, Lucky Sharma  wrote:
> >
> > Hi have one scenario,
> > where I need to make a sequential call to the solr. Both the requests
> > are sequential and output of the first will be required to set some
> > params of the second search request.
> >
> > So for such scenarios, I am writing a plugin which will internally
> > handle both the requests and gives the final response as output.
> >
> > Is this the correct approach?
> > The only assumption I am making here is that both the cores are
> > locally available.
> >
> > --
> > Warm Regards,
> >
> > Lucky Sharma
> > Contact No: +91 9821559918
>

Re: Solr Request Handler

[Alexandre Rafalovitch]

You should not be exposing Solr directly to the client, but treating
it more as a database.

Given that, why would you not write your processing code in that
middle-ware layer?

Regards,
   Alex.
On Mon, 3 Dec 2018 at 06:43, Lucky Sharma  wrote:
>
> Hi have one scenario,
> where I need to make a sequential call to the solr. Both the requests
> are sequential and output of the first will be required to set some
> params of the second search request.
>
> So for such scenarios, I am writing a plugin which will internally
> handle both the requests and gives the final response as output.
>
> Is this the correct approach?
> The only assumption I am making here is that both the cores are
> locally available.
>
> --
> Warm Regards,
>
> Lucky Sharma
> Contact No: +91 9821559918

RE: SolrCloud required ports

[Moshe Recanati | KMS]

Hi Jan,
Thank you.
To summarize we need to open these ports within the cluster:
8983
2181
2888
3888



Regards,
Moshe Recanati
CTO
Mobile  + 972-52-6194481
Skype    :  recanati

More at:  www.kmslh.com | LinkedIn | FB

-Original Message-
From: Jan Høydahl  
Sent: Monday, December 3, 2018 12:43 PM
To: solr-user 
Subject: Re: SolrCloud required ports

Hi

This depends on your exact coniguration, so you should ask the engineers who 
deployed ZK and Solr, not this list.

If default solr port is used, you'd need at least 8983 open between servers and 
from the app server to the cluster.
If default zk port is used, you'd need port 2181 open between all three servers 
but not externally (unless you use a client that needs to talk to zk) Also zk 
needs to communicate internally in the quorum on two other ports, which could 
be using ports 2888 and 3888 but could also be something else depending on your 
exact configs. These will never need to be open outside the cluster.

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 3. des. 2018 kl. 09:22 skrev Moshe Recanati | KMS :
> 
> Hi,
> We're currently running SolrCloud with 3 servers: 3 ZK and 3 Search Engines. 
> Each one on each machine.
> Our security team would like to open only the required ports between the 
> servers.
> Please let me know which ports we need to open between the servers?
>  
> Thank you
>  
> Regards,
> Moshe Recanati
> CTO
> Mobile  + 972-52-6194481 
> Skype:  recanati
>  
>  _news_kms-2Dlighthouse-2Dnamed-2Dgartner-2Dcool-2D121000184.html=DwI
> FaQ=EtlJpXAqSaq3cSC4ACVw6-ifVo6KHbawEuqEp-kfN24=vNaquGtywQ6F1lNXYN
> 9CVw=v6BhW17PQhjSm3ktWyaGQdUkg8AU6Dl5qw-QKMMoTFQ=uMsj8Lg5_yk5C70SF
> Xpo5k1wRp5x-n55rw3x5L4iVwQ=> More at:  www.kmslh.com 
>  | LinkedIn 
>  company_kms-2Dlighthouse=DwIFaQ=EtlJpXAqSaq3cSC4ACVw6-ifVo6KHbawEu
> qEp-kfN24=vNaquGtywQ6F1lNXYN9CVw=v6BhW17PQhjSm3ktWyaGQdUkg8AU6Dl5q
> w-QKMMoTFQ=WCIz8QUF02gSOunmgRennfMTdqBj6llOG0WkXzBurzc=> | FB 
>  _pages_KMS-2Dlighthouse_123774257810917=DwIFaQ=EtlJpXAqSaq3cSC4ACV
> w6-ifVo6KHbawEuqEp-kfN24=vNaquGtywQ6F1lNXYN9CVw=v6BhW17PQhjSm3ktWy
> aGQdUkg8AU6Dl5qw-QKMMoTFQ=QI58JVs9eO7ARCUmSaJ4LVmBnR1unoV0jRSMBFhx7x
> U=>

Solr Request Handler

[Lucky Sharma]

Hi have one scenario,
where I need to make a sequential call to the solr. Both the requests
are sequential and output of the first will be required to set some
params of the second search request.

So for such scenarios, I am writing a plugin which will internally
handle both the requests and gives the final response as output.

Is this the correct approach?
The only assumption I am making here is that both the cores are
locally available.

-- 
Warm Regards,

Lucky Sharma
Contact No: +91 9821559918

Block Join Update

[Lucky Sharma]

Hi,
I have a query regarding block join update,
As far as I know, we cannot update the single doc of a block, we have
to delete the complete block and reindex it again.
Please clarify, If there is something wrong in my understanding.
So for the update in either parent or child, is it recommended to
write our own UpdateProcessor to handle block updates. if we don't
want to send the complete block in the request.

If yes what happens during a race condition.


-- 
Warm Regards,

Lucky Sharma
Contact No :+91 9821559918

Re: Streaming In Solr

[Lucky Sharma]

Hi Joel,
Thanks for the info :)
On Wed, Nov 14, 2018 at 8:13 PM Joel Bernstein  wrote:
>
> The implementation is as follows:
>
> 1) There are "stream sources" that generate results from Solr Cloud
> collections. Some of these include: search, facet, knnSearch, random,
> timeseries, nodes, sql etc...
> 2) There are "stream decorators" that wrap stream sources and operated over
> the result set tuples. Some of these decorators operate over sorted result
> sets and don't need to keep much data in memory. For example the innerJoin
> stream decorator merge joins two sorted streams of tuples. Other stream
> decorators read data into memory and perform operations entirely in memory.
> An example of this is the sort stream decorator.
> 3) The are "stream evaluators" that evaluate expressions over the data.
> This includes math expressions. These expressions can operate in both a
> streaming context using the "select" expression or an in memory context
> using the "let" expression to set variables and operate of vectors and
> matrices in memory.
>
> But basically you can think of it as decorators operating over streams of
> data.
>
>
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Wed, Nov 14, 2018 at 3:26 AM Lucky Sharma  wrote:
>
> > Hi Prakhar,
> > Thanks for the reply, But What I am actually curious to know how is it
> > implemented Internally?
> > On Wed, Nov 14, 2018 at 12:47 PM Prakhar Nigam
> >  wrote:
> > >
> > > HI Lucky Prakhar here
> > >
> > > we have met at a training at Mahindra Comviva.  I have found this
> > article it may be a little helpful
> > >
> > >
> > https://medium.com/@sarkaramrit2/getting-started-with-streaming-expressions-in-apache-solr-b49111a417e3
> > >
> > >
> > >
> > >
> > >
> > > Regards,
> > >
> > > Prakhar
> > >
> > >
> > >
> > > From: Lucky Sharma
> > > Sent: Wednesday, November 14, 2018 9:25 AM
> > > To: solr-user@lucene.apache.org
> > > Subject: Streaming In Solr
> > >
> > >
> > >
> > > Hi I have some doubt regarding how the streaming expressions and
> > > parallel SQL queries are evaluated in SOLR, I tried to dig deep in the
> > > code but wasn't able to find much, A little help will be much
> > > appreciated
> > >
> > > --
> > > Warm Regards,
> > >
> > > Lucky Sharma
> > > Contact No :+91 9821559918
> > > This e-mail and all material transmitted with it are for the use of the
> > intended recipient(s) ONLY and contains confidential information. If you
> > are not the intended recipient, please contact the sender by reply e-mail
> > and destroy all copies and the original message. Any unauthorized review,
> > use, disclosure, dissemination, forwarding, printing or copying of this
> > email or any action taken pursuant to the contents of the present e-mail is
> > strictly prohibited and is unlawful. The recipient acknowledges that
> > Comviva Technologies Limited or its management or directors, are unable to
> > exercise control or ensure the integrity over /of the contents of the
> > information contained in e-mail. Any views expressed herein are those of
> > the individual sender only and no binding nature of the contents shall be
> > implied or assumed unless the sender does so expressly with due authority
> > of Comviva Technologies Limited. E-mail and any contents transmitted with
> > it are prone to viruses and related defects despite all efforts to avoid
> > such by Comviva Technologies Limited.
> >
> >
> >
> > --
> > Warm Regards,
> >
> > Lucky Sharma
> > Contact No :+91 9821559918
> >



-- 
Warm Regards,

Lucky Sharma
Contact No :+91 9821559918

Re: SolrCloud required ports

[Jan Høydahl]

Hi

This depends on your exact coniguration, so you should ask the engineers who 
deployed ZK and Solr, not this list.

If default solr port is used, you'd need at least 8983 open between servers and 
from the app server to the cluster.
If default zk port is used, you'd need port 2181 open between all three servers 
but not externally (unless you use a client that needs to talk to zk)
Also zk needs to communicate internally in the quorum on two other ports, which 
could be using ports 2888 and 3888 but could also be something else depending 
on your exact configs. These will never need to be open outside the cluster.

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 3. des. 2018 kl. 09:22 skrev Moshe Recanati | KMS :
> 
> Hi,
> We're currently running SolrCloud with 3 servers: 3 ZK and 3 Search Engines. 
> Each one on each machine.
> Our security team would like to open only the required ports between the 
> servers.
> Please let me know which ports we need to open between the servers?
>  
> Thank you
>  
> Regards,
> Moshe Recanati
> CTO
> Mobile  + 972-52-6194481 
> Skype:  recanati
>  
> 
> More at:  www.kmslh.com  | LinkedIn 
>  | FB 
> 

Re: solr crashes

[Bernd Fehling]

Hi Danilo,

you have to give more infos about your system and the config.

- 30gb RAM (physical RAM?) how much heap do you have for JAVA?
- how large (in GByte) are your 40 million raw data being indexed?
- how large is your index (in GByte) with 40 million docs indexed?
- which version of Solr and JAVA?
- do you have JAVA garbage collection logs and if so what are they reporting?
- Any FullGC in GC logs?

Regards, Bernd


Am 03.12.18 um 10:09 schrieb Danilo Tomasoni:

Hello all,

We have a configuration with a single node with 30gb of RAM.

We use it to index ~40MLN of documents.

We perform queries with edismax parser that contain often edismax parser 
subqueries with the syntax

'_query_:{!edismax mm=X v=$subqueryN}'

Often X == 1.

This solves the "too many boolean clauses" error we got expanding the query 
terms (often phrase queries) directly in the main query.

Unfortunately in this scenario solr often crashes while performing a query, 
even with a single query and no other source of system load.


Do you have any idea of what's going on here?

Otherwise,

What kind of solr configuration parameters do you think I need to investigate 
first?

What kind of log lines should I search for to understand what's going on?


Thank you

Danilo

solr crashes

[Danilo Tomasoni]

Hello all,

We have a configuration with a single node with 30gb of RAM.

We use it to index ~40MLN of documents.

We perform queries with edismax parser that contain often edismax parser 
subqueries with the syntax


'_query_:{!edismax mm=X v=$subqueryN}'

Often X == 1.

This solves the "too many boolean clauses" error we got expanding the 
query terms (often phrase queries) directly in the main query.


Unfortunately in this scenario solr often crashes while performing a 
query, even with a single query and no other source of system load.



Do you have any idea of what's going on here?

Otherwise,

What kind of solr configuration parameters do you think I need to 
investigate first?


What kind of log lines should I search for to understand what's going on?


Thank you

Danilo

--
Danilo Tomasoni
COSBI

As for the European General Data Protection Regulation 2016/679 on the 
protection of natural persons with regard to the processing of personal data, 
we inform you that all the data we possess are object of treatement in the 
respect of the normative provided for by the cited GDPR.

It is your right to be informed on which of your data are used and how; you may 
ask for their correction, cancellation or you may oppose to their use by 
written request sent by recorded delivery to The Microsoft Research – 
University of Trento Centre for Computational and Systems Biology Scarl, Piazza 
Manifattura 1, 38068 Rovereto (TN), Italy.

SolrCloud required ports

[Moshe Recanati | KMS]

Hi,
We're currently running SolrCloud with 3 servers: 3 ZK and 3 Search Engines. 
Each one on each machine.
Our security team would like to open only the required ports between the 
servers.
Please let me know which ports we need to open between the servers?

Thank you

Regards,
Moshe Recanati
CTO
Mobile  + 972-52-6194481
Skype:  recanati
[KMS2]
More at:  www.kmslh.com | 
LinkedIn | 
FB