The UI warning was quite easy to resolve. I'm currently testing Solr
with HDFS but for some reason the core ended up on the local storage of
the node. After a delete and restart the problem was gone.
On 05.01.2017 12:42, Hendrik Haddorp wrote:
Right, I had to do that multiple times already when I restarted nodes
during collection creation. In such cases I was left with data in the
clusterstate.json, which at least on 6.2.1, blocked further collection
creations. Once manually deleted or set to {} collection creation
worked again.
Setting legacyCloud=false looks good. I don't get anything in
clusterstate.json anymore and no old collections show up after a node
restarts. I could also confirm what Shalin said, that state format 2
is used by default. Only if I explicitly set state format to 1 I see
data in clusterstate.json during the collection creation. Just the
Solr admin UI is now showing "SolrCore Initialization Failures"
pointing to none existing replicas. I assume that happens when Solr
starts up and finds data for a core that does not exist in ZK anymore.
How would one clean up this issue? Beside that the some replicas can
still end up broken if the node restarts in the wrong time. I
currently have one replica marked as down and one as gone. So far I
was however always able to manually replace these replicas to resolve
this state. So in general this looks quite good now. Guess I will
still need to find a way to make sure that I don't restart a node
during collection creation :-(
On 05.01.2017 02:33, Erick Erickson wrote:
Let us know how it goes. You'll probably want to remove the _contents_
of clusterstate.json and just leave it as a pair of brackets , i.e. {}
if for no other reason than it's confusing.
Times past the node needed to be there even if empty. Although I just
tried removing it completely on 6x and I was able to start Solr, part
of the startup process recreates it as an empty node, just a pair of
braces.
Best,
Erick
On Wed, Jan 4, 2017 at 1:22 PM, Hendrik Haddorp
<hendrik.hadd...@gmx.net> wrote:
Hi Erik,
I have actually also seen that behavior already. So will check what
happens when I set that property.
I still believe I'm getting the clusterstate.json set already before
the
node comes up again. But I will try to verify that further tomorrow.
thanks,
Hendrik
On 04/01/17 22:10, Erick Erickson wrote:
Hendrik:
Historically in 4.x, there was code that would reconstruct the
clusterstate.json code. So you would see "deleted" collections come
back. One scenario was:
- Have a Solr node offline that had a replica for a collection.
- Delete that collection
- Bring the node back
- It would register itself in clusterstate.json.
So my guess is that something like this is going on and you're getting
a clusterstate.json that's reconstructed (and possibly not complete).
You can avoid this by specifying legacyCloud=false clusterprop
Kind of a shot in the dark...
Erick
On Wed, Jan 4, 2017 at 11:12 AM, Hendrik Haddorp
<hendrik.hadd...@gmx.net> wrote:
You are right, the code looks like it. But why did I then see
collection
data in the clusterstate.json file? If version 1 is not used I would
assume that no data ends up in there. When explicitly setting the
state
format 2 the system seemed to behave differently. And if the code
always
uses version 2 shouldn't the default in that line be changed
accordingly?
On 04/01/17 16:41, Shalin Shekhar Mangar wrote:
Actually the state format defaults to 2 since many releases (all of
6.x at least). This default is enforced in CollectionsHandler much
before the code in ClusterStateMutator is executed.
On Wed, Jan 4, 2017 at 6:16 PM, Hendrik Haddorp
<hendrik.hadd...@gmx.net> wrote:
Hi,
in
solr-6.3.0/solr/core/src/java/org/apache/solr/cloud/overseer/ClusterStateMutator.java
there is the following code starting line 107:
//TODO default to 2; but need to debug why
BasicDistributedZk2Test fails
early on
String znode = message.getInt(DocCollection.STATE_FORMAT,
1) == 1 ? null
: ZkStateReader.getCollectionPath(cName);
Any if that will be changed to default to version 2 anytime soon?
thanks,
Hendrik
