[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15926810#comment-15926810 ] Varun Thacker commented on SOLR-7191: - > Restarting a single node takes over two minute

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15926204#comment-15926204 ] Joshua Humphries commented on SOLR-7191: Our cluster has many thousands of collecti

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925821#comment-15925821 ] Tim Owen commented on SOLR-7191: Admittedly not thousands of collections, but another anecd

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925790#comment-15925790 ] Jerome Yang commented on SOLR-7191: --- After I test on solr6.4.2 in cloud mode. 32 nodes, o

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893410#comment-15893410 ] Shawn Heisey commented on SOLR-7191: Now that SOLR-10130 has been discovered, there's a

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833804#comment-15833804 ] Damien Kamerman commented on SOLR-7191: --- Regarding the extra threads, I'm thinking th

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833172#comment-15833172 ] Shawn Heisey commented on SOLR-7191: bq. Shawn where can I raise maxThreads of jetty?

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832908#comment-15832908 ] Yago Riveiro commented on SOLR-7191: Restarting a node in 6.3 now takes forever ... I b

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832705#comment-15832705 ] Mark Miller commented on SOLR-7191: --- I know we have gotten pretty good at labeling and gr

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832668#comment-15832668 ] Shawn Heisey commented on SOLR-7191: With hard nproc at 61440, soft nproc at 40960, and

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832664#comment-15832664 ] Shawn Heisey commented on SOLR-7191: I figured I would try the test setup again on Solr

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15702278#comment-15702278 ] Erick Erickson commented on SOLR-7191: -- [~noble.paul] Can we close this since SOLR-728

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357167#comment-15357167 ] Noble Paul commented on SOLR-7191: -- I have simplified this patch and moved it over to SOLR

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357146#comment-15357146 ] Noble Paul commented on SOLR-7191: -- yeah, normally you are fine. If there is a GC pause in

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353556#comment-15353556 ] Erick Erickson commented on SOLR-7191: -- yeah, probably as I'm out of my depth here. Si

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353502#comment-15353502 ] Scott Blum commented on SOLR-7191: -- [~erickerickson] we may be talking about 2 different t

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353437#comment-15353437 ] Erick Erickson commented on SOLR-7191: -- FWIW, I tried both and had no trouble even whe

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353362#comment-15353362 ] Erick Erickson commented on SOLR-7191: -- IIUC, the whole watcher thing is replica based

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353293#comment-15353293 ] Noble Paul commented on SOLR-7191: -- The cluster was very stable when I used dedicated over

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353278#comment-15353278 ] Erick Erickson commented on SOLR-7191: -- Empirically it was fine, but that was on very

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353262#comment-15353262 ] Scott Blum commented on SOLR-7191: -- (for the record, this was on a 5.5.1 based build) > I

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353261#comment-15353261 ] Scott Blum commented on SOLR-7191: -- Paginated getChildren().. always wondered why that was

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353258#comment-15353258 ] Shawn Heisey commented on SOLR-7191: bq. That seems... insane. I am glad that I am not

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15353247#comment-15353247 ] Scott Blum commented on SOLR-7191: -- This may be unrelated to the current patch work, but s

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352431#comment-15352431 ] damien kamerman commented on SOLR-7191: --- Concern based on general principles. e.g. if

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352399#comment-15352399 ] Noble Paul commented on SOLR-7191: -- bq.f there is a collection that has more than coreLoad

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352349#comment-15352349 ] Erick Erickson commented on SOLR-7191: -- Hmmm, is this a concern based on general prin

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352228#comment-15352228 ] damien kamerman commented on SOLR-7191: --- Only coreLoadThreadCount cores are registeri

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352216#comment-15352216 ] Erick Erickson commented on SOLR-7191: -- Noble: A few comments: > the stress setup I

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340698#comment-15340698 ] damien kamerman commented on SOLR-7191: --- The patch from March 2015 was against Solr 4

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340685#comment-15340685 ] Erick Erickson commented on SOLR-7191: -- [~noble.paul] Not sure what version this patch

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340106#comment-15340106 ] Noble Paul commented on SOLR-7191: -- which version is the patch built on? > Improve stabil

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325626#comment-15325626 ] damien kamerman commented on SOLR-7191: --- This fits with what I've seen on solr 4/5. T

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15325575#comment-15325575 ] Erick Erickson commented on SOLR-7191: -- I had to chase after this for a while, so I'm

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970255#comment-14970255 ] Damien Kamerman commented on SOLR-7191: --- After 2min around 100 collections all-green.

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970253#comment-14970253 ] Damien Kamerman commented on SOLR-7191: --- 1. hmmm cancel that. Initially I noticed ver

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14968920#comment-14968920 ] Shalin Shekhar Mangar commented on SOLR-7191: - Thanks Damien. # What is the pu

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14966893#comment-14966893 ] Shawn Heisey commented on SOLR-7191: bq. I created 6,000 collections (3 nodes; 2 x repl

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642990#comment-14642990 ] Shalin Shekhar Mangar commented on SOLR-7191: - bq. Another user on the mailing

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642896#comment-14642896 ] Shawn Heisey commented on SOLR-7191: Another user on the mailing list brought up anothe

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14372212#comment-14372212 ] Shalin Shekhar Mangar commented on SOLR-7191: - bq. The biggest problem I notice

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14372155#comment-14372155 ] Shalin Shekhar Mangar commented on SOLR-7191: - That's awesome, Damien. I'll sta

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368557#comment-14368557 ] Damien Kamerman commented on SOLR-7191: --- Shalin, Another change I made was to cache C

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362511#comment-14362511 ] Shalin Shekhar Mangar commented on SOLR-7191: - This is a very broad issue so th

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358685#comment-14358685 ] Shawn Heisey commented on SOLR-7191: [~dk]: The first thing I thought when I saw that

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358149#comment-14358149 ] Damien Kamerman commented on SOLR-7191: --- With 10K cores I was still seeing many cores

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354965#comment-14354965 ] Shalin Shekhar Mangar commented on SOLR-7191: - bq. I tested 4,000 cores on bran

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354875#comment-14354875 ] Shawn Heisey commented on SOLR-7191: Now that I look closer at that log snippet, it app

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354868#comment-14354868 ] Shawn Heisey commented on SOLR-7191: The branch_5x code is a lot more stable than 5.0,

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354379#comment-14354379 ] Damien Kamerman commented on SOLR-7191: --- I tested 4,000 cores on branch_5x and found

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349849#comment-14349849 ] Shawn Heisey commented on SOLR-7191: Followup on the attached log, because I realized t

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349622#comment-14349622 ] Shawn Heisey commented on SOLR-7191: On the plus side ... in branch_5x, drawing the clo

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349163#comment-14349163 ] Shalin Shekhar Mangar commented on SOLR-7191: - Another issue that will help us

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348836#comment-14348836 ] Shawn Heisey commented on SOLR-7191: As I said above, I started over with a fresh cloud

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348240#comment-14348240 ] Shalin Shekhar Mangar commented on SOLR-7191: - We can optimize the node shutdow

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348235#comment-14348235 ] Shalin Shekhar Mangar commented on SOLR-7191: - Shawn, can you try with branch_5

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348132#comment-14348132 ] Damien Kamerman commented on SOLR-7191: --- I'm more concerned with stability than perfo

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347834#comment-14347834 ] Shawn Heisey commented on SOLR-7191: Regarding jute.maxbuffer: The new stateFormat ens

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347813#comment-14347813 ] Shawn Heisey commented on SOLR-7191: I have thought about this in terms of deciding whe

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347746#comment-14347746 ] Erick Erickson commented on SOLR-7191: -- Just throwing this out there, but tangentially

[jira] [Commented] (SOLR-7191) Improve stability and startup performance of SolrCloud with thousands of collections

[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347689#comment-14347689 ] Shawn Heisey commented on SOLR-7191: At this time I do not know what the actual issues