nickva commented on issue #5879: URL: https://github.com/apache/couchdb/issues/5879#issuecomment-3885811625
> Is the removal of close_on_idle considered permanent going forward, or is there any plan to reintroduce an idle handle eviction mechanism in future releases? There are no plans to bring it back. That way it was implemented had race conditions and didn't work well enough. In fact we disabled it production for years because of issues with it. > Is there an officially recommended upper bound for max_dbs_open for 8GB instances under 3.5? We understand it must now be sized according to memory, but are there general guidelines or tested ranges the project considers safe? The default of 500 is fairly conservative to work on a smaller instances. I'd bump it up to 5000 - 10000 and see what it looks like under concurrent load. As I mentioned for us, often our internal default of 5000 is enough for larger clusters. I don't have exact bytes needed / db handle. You can try a few values with a large number of databases. Scan through them, getting their db info and see how much your node will use. > In your experience, when clusters legitimately require very high distinct DB counts (tens of thousands), is the expectation that users: * Scale` memory proportionally, or *Rely strictly on lower max_dbs_open values and tolerate all_dbs_active errors under pressure? There is a difference between having db handles in the cache and having them active. Your memory should be enough to have them in the cache. It doesn't mean they will all be active. For instance in your case 60k value was clearly not enough to have that many active db handles as 8GB is not memory for it. The nodes would have crashed just like they did when you upgraded to the new version if you had that many active connection at once (active could mean a client connection, or building indexes and other background tasks). Another way to look at, is there is no difference in the capacity of the system before and after the change. In fact with close_on_idle the performance would be worse since the same set of dbs would be opened and closed constantly if they are used on a period just a bit longer than the idle time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
