[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 Nik Everett changed: What|Removed |Added Keywords||upstream Status|NEW |PATCH_TO_REVIEW See Also||https://github.com/facebook ||/hhvm/issues/3999 Assignee|wikibugs-l@lists.wikimedia. |neverett+bugzilla@wikimedia |org |.org --- Comment #15 from Nik Everett --- I believe I have a fix filed upstream. At least, when I apply the fix on my machine my problem goes away. I'm not 100% sure its causing Dan's max_connections issue but it very well could. It was certainly causing some file-like things to stay open when they shouldn't and the MySQL connection falls into that category. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 --- Comment #14 from Dan Duvall --- I'm running into this as well. FWICT, it's not necessarily hitting an open files limit but the mysql max_connections limit (defaults to 151). To reproduce and verify: 1) Disable zend role if enabled and provision 2) Restart mysql or hhvm (or both to be safe) 3) Open mysql cli and `show processlist` (you should see the single mysql process) (19:31) root@localhost:[wiki]> show processlist; ++--+---+--+-+--+---+--+ | Id | User | Host | db | Command | Time | State | Info | ++--+---+--+-+--+---+--+ | 40 | root | localhost | wiki | Query |0 | NULL | show processlist | ++--+---+--+-+--+---+--+ 1 row in set (0.01 sec) 4) Load http://localhost:8080/wiki 5) Rinse and repeat 3 and 4 to view growing number of persistent connections. Once > 151 is reached, you should get the "too many connections" error -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 --- Comment #13 from Nik Everett --- I have the following vagrant roles enabled: apisandbox betafeatures cirrussearch geodata And that triggers it. Zend stops it. You can reproduce it simply by reloading the main page about 40-50 times. It'll start blowing up every time really fast. You can also reproduce it by running the Cirrus test suite against it. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 --- Comment #12 from Nik Everett --- Since I sunk a day into figuring things out I should make it clear what I found before I put this aside. The issue is that hhvm was holding on to all open files and sockets after the request ended. You get about 40 page loads before you run out of file handles. Cirrus's test suite has 1000s of loads so that is why I hit it. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 Nik Everett changed: What|Removed |Added Keywords||hhvm Priority|Unprioritized |High --- Comment #11 from Nik Everett --- Tagging hhvm and setting priority to high. I can no longer develop against hhvm which makes me less confident we'll be able to cut over to it. Since I have no idea why this happens in development I have no idea why it doesn't happen in production. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 --- Comment #10 from Nik Everett --- Looks like enabling the zend role fixes the problem for me. Sadly, I might have to give up on debugging this for a while and use zend. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 --- Comment #9 from Nik Everett --- OK! I smashed my instance (again again) and rebuilt it and it still fails just like this. The trouble is that the hhvm service hangs on to stuff forever. Just never lets go. I commented out 10-CirrusSearch.php and still got the errors so I don't believe Cirrus is itself responsible. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 Nik Everett changed: What|Removed |Added CC||neverett+bugzilla@wikimedia ||.org --- Comment #8 from Nik Everett --- I get it. I'll investigate. It has something to do with hhvm keeping a massive number of open files. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 --- Comment #7 from Bryan Davis --- (In reply to Vogone from comment #6) > Also occurs on Labs-Vagrant, apparently. Is there a way to resolve this > there other than waiting for a few hours? As far as I know, no vagrant > halt/vagrant up exists for Labs-Vagrant. At worst in MediaWiki-Vagrant or Labs you should be able to clear any such errors by restarting the hhvm service(s): sudo service hhvm restart sudo service jobrunner restart I can't think of anything that would hold a db connection open otherwise. Getting to the root of the problem, someone needs to determine if this is an hhvm bug, a general MediaWiki bug or a tuning problem for the default MySQL install we provide. It may very well be that our use-cases have somehow outgrown the default database settings. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 --- Comment #6 from Vogone --- Also occurs on Labs-Vagrant, apparently. Is there a way to resolve this there other than waiting for a few hours? As far as I know, no vagrant halt/vagrant up exists for Labs-Vagrant. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 Prateek Saxena changed: What|Removed |Added CC||psax...@wikimedia.org --- Comment #5 from Prateek Saxena --- I am on zend and I think I get it when I reload a page multiple times in a short period of time. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 --- Comment #4 from Jon --- After switching to zend I have not had this problem all day. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 --- Comment #3 from Jon --- It's happening every hour or so for me. I've switched to zend, will report back my experience later. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 Bryan Davis changed: What|Removed |Added CC||bda...@wikimedia.org --- Comment #2 from Bryan Davis --- Any particular workflow or steps to reproduce? If you can reproduce reliably, I'd be interested to know if `vagrant enable-role zend` (switching wiki runtime from hhvm to php5) makes the problem go away or not. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
[Bug 71370] (Cannot contact the database server: Too many connections ())
https://bugzilla.wikimedia.org/show_bug.cgi?id=71370 Vogone changed: What|Removed |Added CC||leon.liese...@wikipedia.de --- Comment #1 from Vogone --- I experience this problem, too. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ Wikibugs-l mailing list Wikibugs-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
