Re: Replica type affinity

[Jason Gerlowski]

This is a bit of a guess - I haven't used this functionality before.
But to a novice the "tag" Rule Condition for "Rule Based Replica
Placement" sounds similar to the requirements you mentioned above.

https://lucene.apache.org/solr/guide/8_3/rule-based-replica-placement.html#rule-conditions

Good luck,

Jason

On Thu, Jan 30, 2020 at 1:00 PM Karl Stoney
 wrote:
>
> Hey,
> Thanks for the reply but I'm trying to have something fully automated and 
> dynamic.  For context I run solr on kubernetes, and at the moment it works 
> beautifully with autoscaling (i can scale up the kubernetes deployment and 
> solr adds replicas and removes them).
>
> I'm trying to add a new type of node though, backed by very fast but 
> ephemeral disks and the idea was to have only PULL replicas running on those 
> nodes automatically and NRT on the persistent disk instances.
>
> Might be a pipe dream but I'm striving for no manual configuration.
> 
> From: Edward Ribeiro 
> Sent: 30 January 2020 16:56
> To: solr-user@lucene.apache.org 
> Subject: Re: Replica type affinity
>
> Hi Karl,
>
> During collection creation you can specify the `createNodeSet` parameter as
> specified by the Solr Reference Guide snippet below:
>
> "createNodeSet
> Allows defining the nodes to spread the new collection across. The format
> is a comma-separated list of node_names, such as
> localhost:8983_solr,localhost:8984_solr,localhost:8985_solr.
> If not provided, the CREATE operation will create shard-replicas spread
> across all live Solr nodes.
> Alternatively, use the special value of EMPTY to initially create no
> shard-replica within the new collection and then later use the ADDREPLICA
> operation to add shard-replicas when and where required."
>
>
> There's also Collections API that you can use the node parameter of
> ADDREPLICA to specify the node that replica shard should be created on.
> See:
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flucene.apache.org%2Fsolr%2Fguide%2F6_6%2Fcollections-api.html%23CollectionsAPI-Input.9data=02%7C01%7Ckarl.stoney%40autotrader.co.uk%7Ce6f81aab85274cd0081408d7a5a56464%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637160002076345528sdata=3pFUtr6o7vK0srGR60lIUc%2Fo9QSftmAcnQDkcx5z%2Bl8%3Dreserved=0
> Other
> commands that can be useful are REPLACENODE, MOVEREPLICA.
>
> Edward
>
>
> On Thu, Jan 30, 2020 at 1:00 PM Karl Stoney
>  wrote:
>
> > Hey everyone,
> > Does anyone know of a way to have solr replicas assigned to specific nodes
> > by some sort of identifying value (in solrcloud).
> >
> > In summary I’m trying to have some Read only replicas only every be
> > assigned to nodes named “solr-ephemeral-x” and my nrt and masters assigned
> > to “solr-index”.
> >
> > Kind of like rack affinity in elasticsearch!
> >
> > Get Outlook for 
> > iOS<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukefdata=02%7C01%7Ckarl.stoney%40autotrader.co.uk%7Ce6f81aab85274cd0081408d7a5a56464%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637160002076345528sdata=a%2BRpt9TyPy4oksfWZzl79rs7pLIwPnPE4AX%2B2SZr03w%3Dreserved=0>
> > This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office:
> > 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England
> > No. 9439967). This email and any files transmitted with it are confidential
> > and may be legally privileged, and intended solely for the use of the
> > individual or entity to whom they are addressed. If you have received this
> > email in error please notify the sender. This email message has been swept
> > for the presence of computer viruses.
> >
> This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office: 1 
> Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 
> 9439967). This email and any files transmitted with it are confidential and 
> may be legally privileged, and intended solely for the use of the individual 
> or entity to whom they are addressed. If you have received this email in 
> error please notify the sender. This email message has been swept for the 
> presence of computer viruses.

Re: Replica type affinity

[Karl Stoney]

Hey,
Thanks for the reply but I'm trying to have something fully automated and 
dynamic.  For context I run solr on kubernetes, and at the moment it works 
beautifully with autoscaling (i can scale up the kubernetes deployment and solr 
adds replicas and removes them).

I'm trying to add a new type of node though, backed by very fast but ephemeral 
disks and the idea was to have only PULL replicas running on those nodes 
automatically and NRT on the persistent disk instances.

Might be a pipe dream but I'm striving for no manual configuration.

From: Edward Ribeiro 
Sent: 30 January 2020 16:56
To: solr-user@lucene.apache.org 
Subject: Re: Replica type affinity

Hi Karl,

During collection creation you can specify the `createNodeSet` parameter as
specified by the Solr Reference Guide snippet below:

"createNodeSet
Allows defining the nodes to spread the new collection across. The format
is a comma-separated list of node_names, such as
localhost:8983_solr,localhost:8984_solr,localhost:8985_solr.
If not provided, the CREATE operation will create shard-replicas spread
across all live Solr nodes.
Alternatively, use the special value of EMPTY to initially create no
shard-replica within the new collection and then later use the ADDREPLICA
operation to add shard-replicas when and where required."


There's also Collections API that you can use the node parameter of
ADDREPLICA to specify the node that replica shard should be created on.
See:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flucene.apache.org%2Fsolr%2Fguide%2F6_6%2Fcollections-api.html%23CollectionsAPI-Input.9data=02%7C01%7Ckarl.stoney%40autotrader.co.uk%7Ce6f81aab85274cd0081408d7a5a56464%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637160002076345528sdata=3pFUtr6o7vK0srGR60lIUc%2Fo9QSftmAcnQDkcx5z%2Bl8%3Dreserved=0
Other
commands that can be useful are REPLACENODE, MOVEREPLICA.

Edward


On Thu, Jan 30, 2020 at 1:00 PM Karl Stoney
 wrote:

> Hey everyone,
> Does anyone know of a way to have solr replicas assigned to specific nodes
> by some sort of identifying value (in solrcloud).
>
> In summary I’m trying to have some Read only replicas only every be
> assigned to nodes named “solr-ephemeral-x” and my nrt and masters assigned
> to “solr-index”.
>
> Kind of like rack affinity in elasticsearch!
>
> Get Outlook for 
> iOS<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukefdata=02%7C01%7Ckarl.stoney%40autotrader.co.uk%7Ce6f81aab85274cd0081408d7a5a56464%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637160002076345528sdata=a%2BRpt9TyPy4oksfWZzl79rs7pLIwPnPE4AX%2B2SZr03w%3Dreserved=0>
> This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office:
> 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England
> No. 9439967). This email and any files transmitted with it are confidential
> and may be legally privileged, and intended solely for the use of the
> individual or entity to whom they are addressed. If you have received this
> email in error please notify the sender. This email message has been swept
> for the presence of computer viruses.
>
This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office: 1 
Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 
9439967). This email and any files transmitted with it are confidential and may 
be legally privileged, and intended solely for the use of the individual or 
entity to whom they are addressed. If you have received this email in error 
please notify the sender. This email message has been swept for the presence of 
computer viruses.

Re: Replica type affinity

[Edward Ribeiro]

Hi Karl,

During collection creation you can specify the `createNodeSet` parameter as
specified by the Solr Reference Guide snippet below:

"createNodeSet
Allows defining the nodes to spread the new collection across. The format
is a comma-separated list of node_names, such as
localhost:8983_solr,localhost:8984_solr,localhost:8985_solr.
If not provided, the CREATE operation will create shard-replicas spread
across all live Solr nodes.
Alternatively, use the special value of EMPTY to initially create no
shard-replica within the new collection and then later use the ADDREPLICA
operation to add shard-replicas when and where required."


There's also Collections API that you can use the node parameter of
ADDREPLICA to specify the node that replica shard should be created on.
See:
https://lucene.apache.org/solr/guide/6_6/collections-api.html#CollectionsAPI-Input.9
Other
commands that can be useful are REPLACENODE, MOVEREPLICA.

Edward


On Thu, Jan 30, 2020 at 1:00 PM Karl Stoney
 wrote:

> Hey everyone,
> Does anyone know of a way to have solr replicas assigned to specific nodes
> by some sort of identifying value (in solrcloud).
>
> In summary I’m trying to have some Read only replicas only every be
> assigned to nodes named “solr-ephemeral-x” and my nrt and masters assigned
> to “solr-index”.
>
> Kind of like rack affinity in elasticsearch!
>
> Get Outlook for iOS
> This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office:
> 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England
> No. 9439967). This email and any files transmitted with it are confidential
> and may be legally privileged, and intended solely for the use of the
> individual or entity to whom they are addressed. If you have received this
> email in error please notify the sender. This email message has been swept
> for the presence of computer viruses.
>

Replica type affinity

[Karl Stoney]

Hey everyone,
Does anyone know of a way to have solr replicas assigned to specific nodes by 
some sort of identifying value (in solrcloud).

In summary I’m trying to have some Read only replicas only every be assigned to 
nodes named “solr-ephemeral-x” and my nrt and masters assigned to “solr-index”.

Kind of like rack affinity in elasticsearch!

Get Outlook for iOS
This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office: 1 
Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 
9439967). This email and any files transmitted with it are confidential and may 
be legally privileged, and intended solely for the use of the individual or 
entity to whom they are addressed. If you have received this email in error 
please notify the sender. This email message has been swept for the presence of 
computer viruses.