[
https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832664#comment-15832664
]
Shawn Heisey commented on SOLR-7191:
------------------------------------
I figured I would try the test setup again on Solr 6.4, see whether the
situation has improved with newer versions.
The system requirements of thousands of collections has NOT gotten better. It
seems to have gotten considerably worse. The time from node restart to stable
operation MIGHT have improved, but since I haven't yet been able to create all
4000 collections, I cannot be sure about that.
I ran into serious trouble before I had even created 1000 collections. Bumped
the heap and proceeded to create more ... but ran into more trouble. With a
12g heap for the instance running zookeeper, I noticed that I was getting an
OOME about not being able to create threads when I had gotten a little more
than 1800 collections created I have changed nproc in
/etc/security/limits.conf (was a soft limit of 4096 and a hard limit of 6144)
and bumped maxThreads in the Jetty config, and once the cluster is stable after
restart, I will try to make more collections.
> Improve stability and startup performance of SolrCloud with thousands of
> collections
> ------------------------------------------------------------------------------------
>
> Key: SOLR-7191
> URL: https://issues.apache.org/jira/browse/SOLR-7191
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
> Affects Versions: 5.0
> Reporter: Shawn Heisey
> Assignee: Noble Paul
> Labels: performance, scalability
> Fix For: 6.3
>
> Attachments: lots-of-zkstatereader-updates-branch_5x.log,
> SOLR-7191.patch, SOLR-7191.patch, SOLR-7191.patch, SOLR-7191.patch,
> SOLR-7191.patch, SOLR-7191.patch, SOLR-7191.patch
>
>
> A user on the mailing list with thousands of collections (5000 on 4.10.3,
> 4000 on 5.0) is having severe problems with getting Solr to restart.
> I tried as hard as I could to duplicate the user setup, but I ran into many
> problems myself even before I was able to get 4000 collections created on a
> 5.0 example cloud setup. Restarting Solr takes a very long time, and it is
> not very stable once it's up and running.
> This kind of setup is very much pushing the envelope on SolrCloud performance
> and scalability. It doesn't help that I'm running both Solr nodes on one
> machine (I started with 'bin/solr -e cloud') and that ZK is embedded.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
