Re: [Wikitech-l] [Wikivideo-l] Video transcode prioritization changes in progress

[Brion Vibber]

Looks like the high-prio queue is back to normal, all is well. :) Please
continue uploading.

-- brion

On Thu, Feb 9, 2017 at 5:02 PM, James Hare  wrote:

> Should people refrain from uploading videos at this time?
>
>
> On February 9, 2017 at 5:01:51 PM, Brion Vibber (bvib...@wikimedia.org)
> wrote:
>
> Quick update -- I tested some batch-reencoding of missing low-res files and
> have flooded the high-priority queue. :) Thanks for your patience as we get
> the balance right.
>
> I'll add a throttle to TimedMediaHandler's requeueTranscodes.php so I can
> "fire-and-forget" on the background jobs without disturbing things...
>
> -- brion
>
> On Thu, Feb 9, 2017 at 12:00 PM, Brion Vibber 
> wrote:
>
> > Ok, the video scaler queue change is now live: SD and low-resolution
> video
> > output runs on a new, high-priority queue, with HD video output and
> > transcodes of files over 15 minutes in length run on the old queue.
> >
> > Note that existing queued conversions will continue to run on the
> > low-priority queue until it runs out, but most new uploads from here out
> > will have web-ready output much faster.
> >
> > (Load on the servers may be unbalanced for a bit, and may require further
> > tweaking to best balance responsiveness and throughput, so we'll keep an
> > eye on the graphs.)
> >
> > -- brion
> >
> >
> > On Thu, Feb 9, 2017 at 9:39 AM, Brion Vibber 
> > wrote:
> >
> >> We're in progress of deploying a small change to the video scalers[1],
> >> which should improve availability of newly uploaded video and audio
> files.
> >>
> >> The queue will now be split in two, one which covers low-resolution
> >> conversions for relatively short files, and one which covers long files
> and
> >> high-resolution conversions. When there's a flood of large uploads,
> other
> >> new files should still go through the high-priority queue while the
> large
> >> uploads and HD conversions may back up on the low-priority queue.
> >>
> >> The queue-runner side is updated now (done during the 'puppet SWAT'
> >> deployment window), with the MediaWiki side ready to roll around 19:00
> UTC
> >> (11am Pacific time, regular SWAT deployment window).
> >>
> >> [1] https://gerrit.wikimedia.org/r/#/c/336846/
> >>
> >> -- brion
> >>
> >> ___
> >> Wikivideo-l mailing list
> >> wikivide...@lists.wikimedia.org
> >> https://lists.wikimedia.org/mailman/listinfo/wikivideo-l
> >>
> >>
> >
> > ___
> > Wikivideo-l mailing list
> > wikivide...@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikivideo-l
> >
> >
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Wikivideo-l] Video transcode prioritization changes in progress

[James Hare]

Should people refrain from uploading videos at this time?


On February 9, 2017 at 5:01:51 PM, Brion Vibber (bvib...@wikimedia.org)
wrote:

Quick update -- I tested some batch-reencoding of missing low-res files and
have flooded the high-priority queue. :) Thanks for your patience as we get
the balance right.

I'll add a throttle to TimedMediaHandler's requeueTranscodes.php so I can
"fire-and-forget" on the background jobs without disturbing things...

-- brion

On Thu, Feb 9, 2017 at 12:00 PM, Brion Vibber  wrote:

> Ok, the video scaler queue change is now live: SD and low-resolution video
> output runs on a new, high-priority queue, with HD video output and
> transcodes of files over 15 minutes in length run on the old queue.
>
> Note that existing queued conversions will continue to run on the
> low-priority queue until it runs out, but most new uploads from here out
> will have web-ready output much faster.
>
> (Load on the servers may be unbalanced for a bit, and may require further
> tweaking to best balance responsiveness and throughput, so we'll keep an
> eye on the graphs.)
>
> -- brion
>
>
> On Thu, Feb 9, 2017 at 9:39 AM, Brion Vibber 
> wrote:
>
>> We're in progress of deploying a small change to the video scalers[1],
>> which should improve availability of newly uploaded video and audio
files.
>>
>> The queue will now be split in two, one which covers low-resolution
>> conversions for relatively short files, and one which covers long files
and
>> high-resolution conversions. When there's a flood of large uploads, other
>> new files should still go through the high-priority queue while the large
>> uploads and HD conversions may back up on the low-priority queue.
>>
>> The queue-runner side is updated now (done during the 'puppet SWAT'
>> deployment window), with the MediaWiki side ready to roll around 19:00
UTC
>> (11am Pacific time, regular SWAT deployment window).
>>
>> [1] https://gerrit.wikimedia.org/r/#/c/336846/
>>
>> -- brion
>>
>> ___
>> Wikivideo-l mailing list
>> wikivide...@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikivideo-l
>>
>>
>
> ___
> Wikivideo-l mailing list
> wikivide...@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikivideo-l
>
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Wikivideo-l] Video transcode prioritization changes in progress

[Brion Vibber]

Quick update -- I tested some batch-reencoding of missing low-res files and
have flooded the high-priority queue. :) Thanks for your patience as we get
the balance right.

I'll add a throttle to TimedMediaHandler's requeueTranscodes.php so I can
"fire-and-forget" on the background jobs without disturbing things...

-- brion

On Thu, Feb 9, 2017 at 12:00 PM, Brion Vibber  wrote:

> Ok, the video scaler queue change is now live: SD and low-resolution video
> output runs on a new, high-priority queue, with HD video output and
> transcodes of files over 15 minutes in length run on the old queue.
>
> Note that existing queued conversions will continue to run on the
> low-priority queue until it runs out, but most new uploads from here out
> will have web-ready output much faster.
>
> (Load on the servers may be unbalanced for a bit, and may require further
> tweaking to best balance responsiveness and throughput, so we'll keep an
> eye on the graphs.)
>
> -- brion
>
>
> On Thu, Feb 9, 2017 at 9:39 AM, Brion Vibber 
> wrote:
>
>> We're in progress of deploying a small change to the video scalers[1],
>> which should improve availability of newly uploaded video and audio files.
>>
>> The queue will now be split in two, one which covers low-resolution
>> conversions for relatively short files, and one which covers long files and
>> high-resolution conversions. When there's a flood of large uploads, other
>> new files should still go through the high-priority queue while the large
>> uploads and HD conversions may back up on the low-priority queue.
>>
>> The queue-runner side is updated now (done during the 'puppet SWAT'
>> deployment window), with the MediaWiki side ready to roll around 19:00 UTC
>> (11am Pacific time, regular SWAT deployment window).
>>
>> [1] https://gerrit.wikimedia.org/r/#/c/336846/
>>
>> -- brion
>>
>> ___
>> Wikivideo-l mailing list
>> wikivide...@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikivideo-l
>>
>>
>
> ___
> Wikivideo-l mailing list
> wikivide...@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikivideo-l
>
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Wikivideo-l] Video transcode prioritization changes in progress

[Brion Vibber]

Ok, the video scaler queue change is now live: SD and low-resolution video
output runs on a new, high-priority queue, with HD video output and
transcodes of files over 15 minutes in length run on the old queue.

Note that existing queued conversions will continue to run on the
low-priority queue until it runs out, but most new uploads from here out
will have web-ready output much faster.

(Load on the servers may be unbalanced for a bit, and may require further
tweaking to best balance responsiveness and throughput, so we'll keep an
eye on the graphs.)

-- brion


On Thu, Feb 9, 2017 at 9:39 AM, Brion Vibber  wrote:

> We're in progress of deploying a small change to the video scalers[1],
> which should improve availability of newly uploaded video and audio files.
>
> The queue will now be split in two, one which covers low-resolution
> conversions for relatively short files, and one which covers long files and
> high-resolution conversions. When there's a flood of large uploads, other
> new files should still go through the high-priority queue while the large
> uploads and HD conversions may back up on the low-priority queue.
>
> The queue-runner side is updated now (done during the 'puppet SWAT'
> deployment window), with the MediaWiki side ready to roll around 19:00 UTC
> (11am Pacific time, regular SWAT deployment window).
>
> [1] https://gerrit.wikimedia.org/r/#/c/336846/
>
> -- brion
>
> ___
> Wikivideo-l mailing list
> wikivide...@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikivideo-l
>
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l