feliixx opened a new issue, #5585: URL: https://github.com/apache/couchdb/issues/5585
Hi, Thank you for maintaining CouchDB, we've been using it for several years and it's great ! We recently started to migrate our 3 node cluster to CouchDB 3.5.0 (from CouchDB 3.4.3), but we noticed a very large CPU consumption increase on the migrated machine: A typical week with CouchDB 3.4.3 on all nodes:  And the week when we upgraded the first node (`A`) to CouchDB 3.5.0:  (CouchDB was upgraded at around 10 AM on June 30th, it's the first very large spike we see on the graph) - The CPU consumption increase comes only from CouchDB. - All three nodes receives the same amount of traffic. - During the upgrade, we only updated CouchDB, but didn't change anything else on the node. We tried to revert some configuration changes made by CouchDB 3.5.0: - we tried to set `use_cfile` to `false` - we also reverted the changes made to `server_options` by https://github.com/apache/couchdb/pull/5435 But both had no impact. There's nothing unusual in the logs, and the compaction doesn't seems to run more often on the upgraded node. The only difference we spotted is the number of database open. We have a large number of small databases in our cluster (~100 000 databases in total), and we set `max_dbs_open` to `80000`. We noticed that on the v3.5.0 node `open_databases` very quickly reaches the maximum allowed, while on other nodes it remains around ~3500. |node|`open_databases`|`open_os_files`|`message_queues.couch_db_updater.count`| |-|-|-|-| |A (v3.5.0)|80000|86749|79995| |B (v3.4.3)|3526|4685|3526| |C (v3.4.3)|3343|4501|3346| Lowering `max_dbs_open` to `20000` didn't had any impact: the CPU consumption remained higher on the v3.5.0 node. Do you know why we see this large CPU consumption increase with CouchDB 3.5.0 ? And do you know why we see so much more `open_databases` on the upgraded node ? Thanks in advance, and please let me know if you need more information. --- ## Your Environment `curl localhost:5984` ```json { "couchdb": "Welcome", "version": "3.5.0", "git_sha": "11f0d3643", "features": [ "quickjs", "access-ready", "partitioned", "pluggable-storage-engines", "reshard", "scheduler" ] } ``` relevant `localhost:5984/_node/_local/_config`: ```json { "cluster": { "n": "3", "q": "2" }, "couchdb": { "max_attachment_size": "1048576", "max_dbs_open": "20000", "max_document_size": "1048576", "update_lru_on_read": "false", }, "couchdb_engines": { "couch": "couch_bt_engine" }, "feature_flags": { "partitioned||*": "true" }, "indexers": { "couch_mrview": "true" }, "replicator": { "use_checkpoints": "false" }, "smoosh": { "persist": "true", "state_dir": "/var/lib/couchdb" }, "smoosh.ratio_dbs": { "min_size": "131072" }, "smoosh.ratio_views": { "min_size": "131072" }, "smoosh.slack_dbs": { "min_size": "131072" }, "smoosh.slack_views": { "min_size": "131072" } } ``` * Operating system and version: Rocky Linux 9.6 (`Linux 5.14.0-570.21.1.el9_6.x86_64 GNU/Linux`) * Erlang version: `26.2.5.9` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
