[
https://issues.apache.org/jira/browse/SOLR-16751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris M. Hostetter updated SOLR-16751:
--------------------------------------
Attachment: SOLR-16751.patch
Status: Open (was: Open)
I'm not sure why/how this could cause any errors in functionality, since all of
the code I found for _consuming_ {{state.json}} seemed to be doing a "last key
wins" when parsing the JSON, and once a {{DocCollection}} exists, I _think_ all
the consumers are correctly dealing with {{DocCollection.getSlices()}} , and
never looking at (or trying to mutate) a {{"shards"}} key in
{{DocCollection.getProperties()}} ...
...and yet...
We have seen a big spike in recent jenkins failures (with seeds I haven't been
able to reproduce) in
{{SplitShardWithNodeRoleTest.testSolrClusterWithNodeRoleWithPull}} – the timing
of which seems to line up with GIT:2ac7ed29563a33d9f9a31737996a1d4cfb0fca0d.
If i'm right, we should see a big drop in failures from
{{SplitShardWithNodeRoleTest}} if we commit the attached patch.
> ClusterState.collectionFromObjects() stores copy of shards in
> DocCollection.getProperties()
> -------------------------------------------------------------------------------------------
>
> Key: SOLR-16751
> URL: https://issues.apache.org/jira/browse/SOLR-16751
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Chris M. Hostetter
> Priority: Major
> Attachments: SOLR-16751.patch
>
>
> The way a {{DocCollection}} is created by
> {{ClusterState.collectionFromObjects()}} ( ie: {{ClusterState}} is read from
> a JSON payload – such as {{state.json}} ) a copy of the original {{"shards"}}
> mappings is left in the {{props}} argument (where it's carried around in
> {{DocCollection.getProperties()}} ) in addition to the (correct) {{slices}}
> argument (which is used to build {{DocCollection.getSlices()}} )
> This is due to the following lines of code, which AFAICT should have been
> done in reverse order...
> {code:java}
> props = new HashMap<>(objs);
> objs.remove(CollectionStateProps.SHARDS);
> {code}
> This bug seems to have existed for a while, but seems to have slipped under
> the radar since nothing has any reason to be looking for a {{"shards"}} key
> in {{DocCollection.getProperties()}} .
> Recently however, noble committed 2ac7ed29563a33d9f9a31737996a1d4cfb0fca0d
> which changed the serialization logic for {{DocCollection}} to eliminate a
> temporary map...
> {noformat}
> - LinkedHashMap<String, Object> all =
> CollectionUtil.newLinkedHashMap(slices.size() + 1);
> - all.putAll(propMap);
> - all.put(CollectionStateProps.SHARDS, slices);
> - jsonWriter.write(all);
> + jsonWriter.write(
> + (MapWriter)
> + ew -> {
> + propMap.forEach(ew.getBiConsumer());
> + ew.put(CollectionStateProps.SHARDS, slices);
> + });
> {noformat}
> ... so now, instead of an {{all}} Map where the {{"shards"}} key from
> {{propMap}} being (correctly) overridden by {{slices}} and then serialized,
> now the JSON output contains two entries with the {{"shards"}} key.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
