GWicke has uploaded a new change for review. https://gerrit.wikimedia.org/r/192876
Change subject: Lower the heap limit per worker to 250m ...................................................................... Lower the heap limit per worker to 250m tl;dr: Setting the heap threshold to 2x the steady state heap will speed up automatic recovery. In load testing steady state v8 heap usage is actually closer to 120m, and 250m is only reached when cassandra connections are hanging (several nodes going down, quorum being temporarily lost) and cassandra requests are piling up. The next version of the cassandra driver will have reworked timeout and heartbeat handling, which should address the root cause of those hangs. Change-Id: I476e7565c07b781252e21506ac360fcf718bdc78 --- M modules/cdh M modules/restbase/templates/config.yaml.erb 2 files changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet refs/changes/76/192876/1 diff --git a/modules/cdh b/modules/cdh index 5dcfc4d..2d5a1d5 160000 --- a/modules/cdh +++ b/modules/cdh -Subproject commit 5dcfc4d029b2ee9fe6d4c416b0a6b1bc6d7ef111 +Subproject commit 2d5a1d5e30f70e1e49560d57c9cd435102864d20 diff --git a/modules/restbase/templates/config.yaml.erb b/modules/restbase/templates/config.yaml.erb index 7f2c4a6..b7c1fb3 100644 --- a/modules/restbase/templates/config.yaml.erb +++ b/modules/restbase/templates/config.yaml.erb @@ -6,7 +6,7 @@ # Log error messages and gracefully restart a worker if v8 reports using more # heap (note: not RSS). -worker_heap_limit_mb: 500 +worker_heap_limit_mb: 250 logging: name: restbase -- To view, visit https://gerrit.wikimedia.org/r/192876 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I476e7565c07b781252e21506ac360fcf718bdc78 Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet Gerrit-Branch: production Gerrit-Owner: GWicke <[email protected]> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
