Nicely done, Brion! We’re very grateful for all the mobile multimedia work you’ve been doing on your ‘spare time’ …
Much appreciated, Fabrice > On Jun 11, 2015, at 7:34 AM, Brion Vibber <bvib...@wikimedia.org> wrote: > > I've been passing the last few days feverishly working on audio/video stuff, > cause it's been driving me nuts that it's not quite in working shape. > > TL;DR: Major fixes in the works for Android, Safari (iOS and Mac), and > IE/Edge (Windows). Need testers and patch reviewers. > > > == ogv.js for Safari/IE/Edge == > > In recent versions of Safari, Internet Explorer, and Microsoft's upcoming > Edge browser, there's still no default Ogg or WebM support but JavaScript has > gotten fast enough to run an Ogg Theora/Vorbis decoder with CPU to spare for > drawing and outputting sound in real time. > > The ogv.js decoder/player has been one of my fun projects for some time, and > I think I'm finally happy with my TimedMediaHandler/MwEmbedPlayer integration > patch <https://gerrit.wikimedia.org/r/#/c/165478/> for the desktop MediaWiki > interface. > > I'll want to update it to work with Video.js later, but I'd love to get this > version reviewed and deployed in the meantime. > > Please head over to https://ogvjs-testing.wmflabs.org/ > <https://ogvjs-testing.wmflabs.org/> in Safari 6.1+ or IE 10+ (or 'Project > Spartan' on Windows 10 preview) and try it out! Particularly interested in > cases where it doesn't work or messes up. > > > == Non-JavaScript fallback for iOS == > > I've found that Safari on iOS supports QuickTime movies with Motion-JPEG > video and mu-law PCM audio <https://gerrit.wikimedia.org/r/#/c/217295/>. JPEG > and PCM are, as it happens, old and not so much patented. \o/ > > As such this should work as a fallback for basic audio and video on older > iPhones and iPads that can't run ogv.js well, or in web views in apps that > use Apple's older web embedding APIs where JavaScript is slow (for example, > Chrome for iOS). > > However these get really bad compression ratios, so to keep bandwidth down > similar to the 360p Ogg and WebM versions I had to reduce quality and > resolution significantly. Hold an iPhone at arm's length and it's maybe ok, > but zoom full-screen on your iPad and you'll hate the giant blurry pixels! > > This should also provide a working basic audio/video experience in our > Wikipedia iOS app, until such time as we integrate Ogg or WebM decoding > natively into the app. > > Note that it seems tricky to bulk-run new transcodes on old files with > TimedMediaHandler. I assume there's a convenient way to do it that I just > haven't found in the extension maint scripts... > > > == In progress: mobile video fixes == > > Audio has worked on Android for a while -- the .ogg files show up in native > <audio> elements and Just Work. > > But video has been often broken, with TimedMediaHandler's "popup transforms" > reducing most video embeds into a thumbnail and a link to the original file > -- which might play if WebM (not if Ogg Theora) but it might also be a 1080p > original which you don't want to pull down on 3G! And neither audio nor video > has worked on iOS. > > This patch <https://gerrit.wikimedia.org/r/#/c/217485/> adds a simple mobile > target for TMH, which fixes the popup transforms to look better and actually > work by loading up an embedded-size player with the appropriately playable > transcodes (WebM, Ogg, and the MJPEG last-ditch fallback). > > ogv.js is used if available and necessary, for instance in iOS Safari when > the CPU is fast enough. (Known to work only on 64-bit models.) > > > == Future: codec.js and WebM and OGVKit == > > For the future, I'm also working on extending ogv.js to support WebM > <https://brionv.com/log/2015/06/07/im-in-ur-javascript-decoding-ur-webm/> for > better quality (especially in high-motion scenes) -- once that stabilizes > I'll rename the combined package codec.js. Performance of WebM is not yet > good enough to deploy, and some features like seeking are still missing, but > breaking out the codec modules means I can develop the codecs in parallel and > keep the high-level player logic in common. > > Browser infrastructure improvements like SIMD, threading, and more GPU access > should continue to make WebM decoding faster in the future as well. > > > I'd also like to finish up my OGVKit package > <https://github.com/brion/OGVKit> for iOS, so we can embed a basic > audio/video player at full quality into the Wikipedia iOS app. This needs > some more cleanup work still. > > > Phew! Ok that's about it. > > -- brion > _______________________________________________ > Multimedia mailing list > multime...@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/multimedia _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
