Brion VIBBER has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/333035 )
Change subject: Double $wgTranscodeBackgroundTimeLimit to compensate for
threading
......................................................................
Double $wgTranscodeBackgroundTimeLimit to compensate for threading
Video transcodes of very long files at very high resolutions were
often getting a SIGKILL (exit code 137) but much earlier than
expected based on the time and memory limits.
Default time limit of 8 hours was being reached in 4-6 hours due
to threading in ffmpeg, which hit them around 175-185% CPU instead
of a 'mere' 100%. Math is hard!
Doubling the limit will bring it into line with expectations, and
should avoid wasting the CPU time of the first encoding by forcing
a manual second/third/etc re-encode.
This means the final timeout will hit around 8-12 hours, which should
cover all legit expected cases while still killing infinite-broken
processes.
Bug: T155750
Change-Id: I21aa914d2ea64cf345b1ae5ec1fbc3be8e7d4902
---
M wmf-config/CommonSettings.php
1 file changed, 7 insertions(+), 0 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config
refs/changes/35/333035/1
diff --git a/wmf-config/CommonSettings.php b/wmf-config/CommonSettings.php
index 15de867..c5d7aa6 100644
--- a/wmf-config/CommonSettings.php
+++ b/wmf-config/CommonSettings.php
@@ -849,6 +849,13 @@
//for avconv / ffmpeg2theora
$wgTranscodeBackgroundMemoryLimit = 4 * 1024 * 1024; // 4GB
$wgFFmpegThreads = 2;
+
+ // ffmpeg tends to use about 175% CPU when threaded, so hits
+ // the default 8-hour ulimit in 4-6 hours. This tends to cut
+ // off very large files at very high resolution just before
+ // they finish, wasting a lot of time.
+ // Pad it back out so we don't waste that CPU time with a fail!
+ $wgTranscodeBackgroundTimeLimit *= $wgFFmpegThreads;
// Minimum size for an embed video player
$wgMinimumVideoPlayerSize = $wmgMinimumVideoPlayerSize;
--
To view, visit https://gerrit.wikimedia.org/r/333035
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I21aa914d2ea64cf345b1ae5ec1fbc3be8e7d4902
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Brion VIBBER <br...@wikimedia.org>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
