Ah... the multiple shards (of the same collection) in a single node is about
planning for future expansion of your cluster - create more shards than you
need today, put more of them on a single node and then migrate them to their
own nodes as the data outgrows the smaller number of nodes. In other words,
add nodes incrementally without having to reindex all the data.
-- Jack Krupansky
-----Original Message-----
From: Darren Govoni
Sent: Thursday, January 03, 2013 9:18 AM
To: solr-user@lucene.apache.org
Subject: RE: Re: Terminology question: Core vs. Collection vs...
Yes. And its worth to note that when having multiple shards in a single
node(@deprecated) that they are shards of different collections...
<br><br><br>------- Original Message -------
On 1/3/2013 09:16 AM Jack Krupansky wrote:<br>And I would revise "node" to
note that in SolrCloud a node is simply an
<br>instance of a Solr server.
<br>
<br>And, technically, you can have multiple shards in a single instance of
Solr,
<br>separating the logical sharding of keys from the distribution of the
data.
<br>
<br>-- Jack Krupansky
<br>
<br>-----Original Message-----
<br>From: Jack Krupansky
<br>Sent: Thursday, January 03, 2013 9:08 AM
<br>To: solr-user@lucene.apache.org
<br>Subject: Re: Terminology question: Core vs. Collection vs...
<br>
<br>Oops... let me word that a little more carefully:
<br>
<br>...we are "replicating the data of each shard".
<br>
<br>
<br>
<br>
<br>
<br>-- Jack Krupansky
<br>-----Original Message-----
<br>From: Jack Krupansky
<br>Sent: Thursday, January 03, 2013 9:03 AM
<br>To: solr-user@lucene.apache.org
<br>Subject: Re: Terminology question: Core vs. Collection vs...
<br>
<br>No, a shard is a subset (or "slice") of the collection. Sharding is a
way of
<br>"slicing" the original data, before we talk about how the shards get
stored
<br>and replicated on actual Solr cores. Replicas are instances of the data
for
<br>a shard.
<br>
<br>Sometimes people may loosely speak of a replica as being "a shard", but
<br>that's just loose use of the terminology.
<br>
<br>So, we're not "sharding shards", but we are "replicating shards".
<br>
<br>-- Jack Krupansky
<br>
<br>-----Original Message-----
<br>From: Darren Govoni
<br>Sent: Thursday, January 03, 2013 8:51 AM
<br>To: solr-user@lucene.apache.org
<br>Subject: RE: Re: Terminology question: Core vs. Collection vs...
<br>
<br>Thanks again. (And sorry to jump into this convo)
<br>
<br>But I had a question on your statement:
<br>
<br>On 1/3/2013 08:07 AM Jack Krupansky wrote:
<br> <br>Collection is the more modern term and incorporates the fact that
the
<br><br>collection may be sharded, with each shard on one or more cores,
with
<br>each <br>core being a replica of the other cores within that shard of
that
<br><br>collection.
<br>
<br>A collection is sharded, meaning it is distributed across cores. A shard
<br>itself is not distributed across cores in the same since. Rather a shard
<br>exist on a single core and is replicated on other cores. Is that right?
The
<br>way its worded above, it sounds like a shard can also be sharded...
<br>
<br>
<br><br><br><br>------- Original Message -------
<br>On 1/3/2013 08:28 AM Jack Krupansky wrote:<br>A node is a machine in a
<br>cluster or cloud (graph). It could be a real
<br><br>machine or a virtualized machine. Technically, you could have
multiple
<br><br>virtual nodes on the same physical "box". Each Solr replica would be
on
<br>a
<br><br>different node.
<br><br>
<br><br>Technically, you could have multiple Solr instances running on a
single
<br><br>hardware node, each with a different port. They are simply instances
of
<br><br>Solr, although you could consider each Solr instance a node in a
Solr
<br>cloud
<br><br>as well, a "virtual" node. So, technically, you could have multiple
<br>replicas
<br><br>on the same node, but that sort of defeats most of the purpose of
having
<br><br>replicas in the first place - to distribute the data for performance
and
<br><br>fault tolerance. But, you could have replicas of different shards on
the
<br><br>same node/box for a partial improvement of performance and fault
<br>tolerance.
<br><br>
<br><br>A Solr "cloud' is really a cluster.
<br><br>
<br><br>-- Jack Krupansky
<br><br>
<br><br>-----Original Message-----
<br><br>From: Darren Govoni
<br><br>Sent: Thursday, January 03, 2013 8:16 AM
<br><br>To: solr-user@lucene.apache.org
<br><br>Subject: RE: Re: Terminology question: Core vs. Collection vs...
<br><br>
<br><br>Good write up.
<br><br>
<br><br>And what about "node"?
<br><br>
<br><br>I think there needs to be an official glossary of terms that is
<br>sanctioned
<br><br>by the solr team and some terms still ni use may need to be labeled
<br><br>"deprecated". After so many years, its still confusing.
<br><br>
<br><br><br><br><br>------- Original Message -------
<br><br>On 1/3/2013 08:07 AM Jack Krupansky wrote:<br>Collection is the
more
<br>modern
<br><br>term and incorporates the fact that the
<br><br><br>collection may be sharded, with each shard on one or more cores,
<br>with
<br><br>each
<br><br><br>core being a replica of the other cores within that shard of
that
<br><br><br>collection.
<br><br><br>
<br><br><br>Instance is a general term, but is commonly used to refer to a
<br>running
<br><br>Solr
<br><br><br>server, each of which can service any number of cores. A sharded
<br><br>collection
<br><br><br>would typically require multiple instances of Solr, each with a
<br>shard of
<br><br>the
<br><br><br>collection.
<br><br><br>
<br><br><br>Multiple collections can be supported on a single instance of
Solr.
<br>They
<br><br><br>don't have to be sharded or replicated. But if they are, each
Solr
<br><br>instance
<br><br><br>will have a copy or replica of the data (index) of one shard of
each
<br><br>sharded
<br><br><br>collection - to the degree that each collection needs that many
<br>shards.
<br><br><br>
<br><br><br>At the API level, you talk to a Solr instance, using a host and
<br>port,
<br><br>and
<br><br><br>giving the collection name. Some operations will refer only to
the
<br><br>portion
<br><br><br>of a multi-shard collection on that Solr instance, but typically
<br>Solr
<br><br>will
<br><br><br>"distribute" the operation, whether it be an update or a query,
to
<br>all
<br><br>of
<br><br><br>the shards of the named collection. In the case of update, the
<br>update
<br><br>will
<br><br><br>be distributed to all replicas as well, but in the case of query
<br>only
<br><br>one
<br><br><br>replica of each shard of the collection is needed.
<br><br><br>
<br><br><br>Before SolrCloud we Solr had master and slave and the slaves
were
<br><br>replicas
<br><br><br>of the master, but with SolrCloud there is no master and all the
<br><br>replicas of
<br><br><br>the shard are peers, although at any moment of time one of them
will
<br>be
<br><br><br>considered the "leader" for coordination purposes, but not in
the
<br>sense
<br><br>that
<br><br><br>it is a master of the other replicas in that shard. A SolrCloud
<br>replica
<br><br>is a
<br><br><br>replica of the data, in an abstract sense, for a single shard of
a
<br><br><br>collection. A SolrCloud replica is more of an instance of the
<br><br>data/index.
<br><br><br>
<br><br><br>An index exists at two levels: the portion of a collection on a
<br>single
<br><br>Solr
<br><br><br>core will have a Lucene index, but collectively the Lucene
indexes
<br>for
<br><br>the
<br><br><br>shards of a collection can be referred to the index of the
<br>collection.
<br><br>Each
<br><br><br>replica is a copy or instance of a portion of the collection's
<br>index.
<br><br><br>
<br><br><br>The term slice is sometimes used to refer collectively to all of
the
<br><br><br>cores/replicas of a single shard, or sometimes to a single
replica
<br>as it
<br><br><br>contains only a "slice" of the full collection data.
<br><br><br>
<br><br><br>-- Jack Krupansky
<br><br><br>
<br><br><br>-----Original Message-----
<br><br><br>From: Alexandre Rafalovitch
<br><br><br>Sent: Thursday, January 03, 2013 4:42 AM
<br><br><br>To: solr-user@lucene.apache.org
<br><br><br>Subject: Terminology question: Core vs. Collection vs...
<br><br><br>
<br><br><br>Hello,
<br><br><br>
<br><br><br>I am trying to understand the core Solr terminology. I am
looking
<br>for
<br><br><br>correct rather than loose meaning as I am trying to teach an
example
<br><br>that
<br><br><br>starts from easy scenario and may scale to multi-core,
multi-machine
<br><br><br>situation.
<br><br><br>
<br><br><br>Here are the terms that seem to be all overlapping and/or
crossing
<br>over
<br><br>in
<br><br><br>my mind a the moment.
<br><br><br>
<br><br><br>1) Index
<br><br><br>2) Core
<br><br><br>3) Collection
<br><br><br>4) Instance
<br><br><br>5) Replica (Replica of _what_?)
<br><br><br>6) Others?
<br><br><br>
<br><br><br>I tried looking through documentation, but either there is a
<br>terminology
<br><br><br>drift or I am having trouble understanding the distinctions.
<br><br><br>
<br><br><br>If anybody has a clear picture in their mind, I would appreciate
a
<br><br><br>clarification.
<br><br><br>
<br><br><br>Regards,
<br><br><br> Alex.
<br><br><br>
<br><br><br>Personal blog: http://blog.outerthoughts.com/
<br><br><br>LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
<br><br><br>- Time is the quality of nature that keeps events from happening
all
<br>at
<br><br><br>once. Lately, it doesn't seem to be working. (Anonymous - via
GTD
<br><br>book)
<br><br><br>
<br><br><br>
<br><br>
<br><br>
<br>
<br>