[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Sam Reed (reedy) changed: What|Removed |Added Priority|Unprioritized |High Summary|Use more th

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #1 from Jan Gerber --- problem with using multiple threads is that the memory use per encode multiplies too. whats the amount of ram available in the box, we might be able to use 2 threads per encode now, the first set of encoding b

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #2 from Brion Vibber --- If y'all have to buy more RAM, please buy more RAM. :) Leaving CPUs idle while transcoding a 40m-minute 1080p video makes no sense at all. -- You are receiving this mail because: You are on the CC list fo

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #3 from Sam Reed (reedy) --- reedy@tmh1001:~$ cat /proc/meminfo MemTotal: 16419920 kB so 16GB See also http://ganglia.wikimedia.org/latest/?r=hour&cs=&ce=&s=by+name&c=Video%2520scalers%2520eqiad&tab=m&vn= Looks like we're

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #4 from Sam Reed (reedy) --- I suspect we could at least double that one to a 2. The jobs are done synchronously, right? ie one at a time (per job runner)? In which case, we don't care so much how much memory is used at a time --

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #5 from Sam Reed (reedy) --- Thinking about it, the simplest way to fix this bug might be to just make a global. It can then be set for WMF usage as appropriate - even (number of cores - 1) if the boxes only do one at a time. Code

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #6 from Jan Gerber --- right now we run 10 concurrent jobs(puppet manifests/role/applicationserver.pp) with $wgTranscodeBackgroundMemoryLimit set to 2GB thats 20GB ram. Since a lot of this is mmapped this is working out. Setting thr

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Gerrit Notification Bot changed: What|Removed |Added Status|NEW |PATCH_TO_REVIEW -- You are

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #7 from Gerrit Notification Bot --- Change 84493 had a related patch set uploaded by J: Make number of threads a configuration option https://gerrit.wikimedia.org/r/84493 -- You are receiving this mail because: You are on the CC

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #8 from Sam Reed (reedy) --- (In reply to comment #6) > right now we run 10 concurrent jobs(puppet > manifests/role/applicationserver.pp) > with $wgTranscodeBackgroundMemoryLimit set to 2GB thats 20GB ram. Since a lot > of this is m

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #9 from Gerrit Notification Bot --- Change 84539 had a related patch set uploaded by J: Set $wgFFmpegThreads to 2 for faster transcodes https://gerrit.wikimedia.org/r/84539 -- You are receiving this mail because: You are on the C

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #10 from Gerrit Notification Bot --- Change 84493 merged by jenkins-bot: Make number of threads a configuration option https://gerrit.wikimedia.org/r/84493 -- You are receiving this mail because: You are on the CC list for the bu

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Gerrit Notification Bot changed: What|Removed |Added Status|RESOLVED|PATCH_TO_REVIEW Reso

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Sam Reed (reedy) changed: What|Removed |Added Status|PATCH_TO_REVIEW |RESOLVED Resolution|---

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #11 from Gerrit Notification Bot --- Change 84775 had a related patch set uploaded by Reedy: Drop Video Scalers to 5 concurrent jobs https://gerrit.wikimedia.org/r/84775 -- You are receiving this mail because: You are on the CC l

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #12 from Gerrit Notification Bot --- Change 84775 merged by Akosiaris: Drop Video Scalers to 5 concurrent jobs https://gerrit.wikimedia.org/r/84775 -- You are receiving this mail because: You are on the CC list for the bug. _

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #13 from Gerrit Notification Bot --- Change 84539 merged by jenkins-bot: Set $wgFFmpegThreads to 2 for faster transcodes https://gerrit.wikimedia.org/r/84539 -- You are receiving this mail because: You are on the CC list for the

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Rob Lanphier changed: What|Removed |Added CC||ro...@wikimedia.org Assignee|

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 --- Comment #14 from Andre Klapper --- So is more work needed here (if so: what?) or can this be closed as RESOLVED FIXED? -- You are receiving this mail because: You are on the CC list for the bug. ___

[Bug 54060] Use more than 1 thread per transcode

https://bugzilla.wikimedia.org/show_bug.cgi?id=54060 Jan Gerber changed: What|Removed |Added Status|PATCH_TO_REVIEW |RESOLVED Resolution|---