[WikimediaMobile] Audio/video updates: TimedMediaHandler, ogv.js, and mobile

[Brion Vibber]

I've been passing the last few days feverishly working on audio/video
stuff, cause it's been driving me nuts that it's not quite in working shape.

TL;DR: Major fixes in the works for Android, Safari (iOS and Mac), and
IE/Edge (Windows). Need testers and patch reviewers.


*== ogv.js for Safari/IE/Edge ==*

In recent versions of Safari, Internet Explorer, and Microsoft's upcoming
Edge browser, there's still no default Ogg or WebM support *but* JavaScript
has gotten fast enough to run an Ogg Theora/Vorbis decoder with CPU to
spare for drawing and outputting sound in real time.

The ogv.js decoder/player has been one of my fun projects for some time,
and I think I'm finally happy with my TimedMediaHandler/MwEmbedPlayer
integration patch  for the
desktop MediaWiki interface.

I'll want to update it to work with Video.js later, but I'd love to get
this version reviewed and deployed in the meantime.

Please head over to https://ogvjs-testing.wmflabs.org/ in Safari 6.1+ or IE
10+ (or 'Project Spartan' on Windows 10 preview) and try it out!
Particularly interested in cases where it doesn't work or messes up.


*== Non-JavaScript fallback for iOS ==*

I've found that Safari on iOS supports QuickTime movies with Motion-JPEG
video and mu-law PCM audio .
JPEG and PCM are, as it happens, old and not so much patented. \o/

As such this should work as a fallback for basic audio and video on older
iPhones and iPads that can't run ogv.js well, or in web views in apps that
use Apple's older web embedding APIs where JavaScript is slow (for example,
Chrome for iOS).

However these get really bad compression ratios, so to keep bandwidth down
similar to the 360p Ogg and WebM versions I had to reduce quality and
resolution significantly. Hold an iPhone at arm's length and it's maybe ok,
but zoom full-screen on your iPad and you'll hate the giant blurry pixels!

This should also provide a working basic audio/video experience in our
Wikipedia iOS app, until such time as we integrate Ogg or WebM decoding
natively into the app.

Note that it seems tricky to bulk-run new transcodes on old files with
TimedMediaHandler. I assume there's a convenient way to do it that I just
haven't found in the extension maint scripts...


*== In progress: mobile video fixes ==*

Audio has worked on Android for a while -- the .ogg files show up in native
 elements and Just Work.

But video has been often broken, with TimedMediaHandler's "popup
transforms" reducing most video embeds into a thumbnail and a link to the
original file -- which might play if WebM (not if Ogg Theora) but it might
also be a 1080p original which you don't want to pull down on 3G! And
neither audio nor video has worked on iOS.

This patch  adds a simple
mobile target for TMH, which fixes the popup transforms to look better and
actually work by loading up an embedded-size player with the appropriately
playable transcodes (WebM, Ogg, and the MJPEG last-ditch fallback).

ogv.js is used if available and necessary, for instance in iOS Safari when
the CPU is fast enough. (Known to work only on 64-bit models.)


*== Future: codec.js and WebM and OGVKit ==*

For the future, I'm also working on extending ogv.js to support WebM

for better quality (especially in high-motion scenes) -- once that
stabilizes I'll rename the combined package codec.js. Performance of WebM
is not yet good enough to deploy, and some features like seeking are still
missing, but breaking out the codec modules means I can develop the codecs
in parallel and keep the high-level player logic in common.

Browser infrastructure improvements like SIMD, threading, and more GPU
access should continue to make WebM decoding faster in the future as well.


I'd also like to finish up my OGVKit package
 for iOS, so we can embed a basic
audio/video player at full quality into the Wikipedia iOS app. This needs
some more cleanup work still.


Phew! Ok that's about it.

-- brion
___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

Re: [WikimediaMobile] [Multimedia] Audio/video updates: TimedMediaHandler, ogv.js, and mobile

[Fabrice Florin]

Nicely done, Brion!

We’re very grateful for all the mobile multimedia work you’ve been doing on 
your ‘spare time’ …

Much appreciated,


Fabrice


> On Jun 11, 2015, at 7:34 AM, Brion Vibber  wrote:
> 
> I've been passing the last few days feverishly working on audio/video stuff, 
> cause it's been driving me nuts that it's not quite in working shape.
> 
> TL;DR: Major fixes in the works for Android, Safari (iOS and Mac), and 
> IE/Edge (Windows). Need testers and patch reviewers.
> 
> 
> == ogv.js for Safari/IE/Edge ==
> 
> In recent versions of Safari, Internet Explorer, and Microsoft's upcoming 
> Edge browser, there's still no default Ogg or WebM support but JavaScript has 
> gotten fast enough to run an Ogg Theora/Vorbis decoder with CPU to spare for 
> drawing and outputting sound in real time.
> 
> The ogv.js decoder/player has been one of my fun projects for some time, and 
> I think I'm finally happy with my TimedMediaHandler/MwEmbedPlayer integration 
> patch  for the desktop MediaWiki 
> interface.
> 
> I'll want to update it to work with Video.js later, but I'd love to get this 
> version reviewed and deployed in the meantime.
> 
> Please head over to https://ogvjs-testing.wmflabs.org/ 
>  in Safari 6.1+ or IE 10+ (or 'Project 
> Spartan' on Windows 10 preview) and try it out! Particularly interested in 
> cases where it doesn't work or messes up.
> 
> 
> == Non-JavaScript fallback for iOS ==
> 
> I've found that Safari on iOS supports QuickTime movies with Motion-JPEG 
> video and mu-law PCM audio . JPEG 
> and PCM are, as it happens, old and not so much patented. \o/
> 
> As such this should work as a fallback for basic audio and video on older 
> iPhones and iPads that can't run ogv.js well, or in web views in apps that 
> use Apple's older web embedding APIs where JavaScript is slow (for example, 
> Chrome for iOS).
> 
> However these get really bad compression ratios, so to keep bandwidth down 
> similar to the 360p Ogg and WebM versions I had to reduce quality and 
> resolution significantly. Hold an iPhone at arm's length and it's maybe ok, 
> but zoom full-screen on your iPad and you'll hate the giant blurry pixels!
> 
> This should also provide a working basic audio/video experience in our 
> Wikipedia iOS app, until such time as we integrate Ogg or WebM decoding 
> natively into the app.
> 
> Note that it seems tricky to bulk-run new transcodes on old files with 
> TimedMediaHandler. I assume there's a convenient way to do it that I just 
> haven't found in the extension maint scripts...
> 
> 
> == In progress: mobile video fixes ==
> 
> Audio has worked on Android for a while -- the .ogg files show up in native 
>  elements and Just Work.
> 
> But video has been often broken, with TimedMediaHandler's "popup transforms" 
> reducing most video embeds into a thumbnail and a link to the original file 
> -- which might play if WebM (not if Ogg Theora) but it might also be a 1080p 
> original which you don't want to pull down on 3G! And neither audio nor video 
> has worked on iOS.
> 
> This patch  adds a simple mobile 
> target for TMH, which fixes the popup transforms to look better and actually 
> work by loading up an embedded-size player with the appropriately playable 
> transcodes (WebM, Ogg, and the MJPEG last-ditch fallback).
> 
> ogv.js is used if available and necessary, for instance in iOS Safari when 
> the CPU is fast enough. (Known to work only on 64-bit models.)
> 
> 
> == Future: codec.js and WebM and OGVKit ==
> 
> For the future, I'm also working on extending ogv.js to support WebM 
>  for 
> better quality (especially in high-motion scenes) -- once that stabilizes 
> I'll rename the combined package codec.js. Performance of WebM is not yet 
> good enough to deploy, and some features like seeking are still missing, but 
> breaking out the codec modules means I can develop the codecs in parallel and 
> keep the high-level player logic in common.
> 
> Browser infrastructure improvements like SIMD, threading, and more GPU access 
> should continue to make WebM decoding faster in the future as well.
>  
> 
> I'd also like to finish up my OGVKit package 
>  for iOS, so we can embed a basic 
> audio/video player at full quality into the Wikipedia iOS app. This needs 
> some more cleanup work still.
> 
> 
> Phew! Ok that's about it.
> 
> -- brion
> ___
> Multimedia mailing list
> multime...@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/multimedia

___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

[WikimediaMobile] iOS Beta 4.1.5 available for testing

[Corey Floyd]

Hey everyone, we have a new iOS release in the hopper. If you are part of
the beta channel, you can download the new version using Apple's TestFlight.

If you would like to be in the beta, just email me, Brian or Monte to get
access.

Here are the highlights of this release:

* Lead images now load faster and fade in smoothly
* More article layout improvements
* More reliable loading of articles from the cache
* Added a link to the FAQ in Settings > More
* Fix crash on startup for some device languages (Like Norwegian)
* Several other crash fixes

Also of note: this our last "scheduled" iOS 6 release. After release, this
version will be available for download indefinitely for our users who are
unable to upgrade to iOS 7.

For users who are running  iOS 7 or higher, this will allow us to support
more great features in future versions of the iOS application.

Thanks for testing and please send us any feedback you may have!

-- 
Corey Floyd
Software Engineer
Mobile Apps / iOS
Wikimedia Foundation
___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

Re: [WikimediaMobile] [Multimedia] Audio/video updates: TimedMediaHandler, ogv.js, and mobile

[Michael Dale]

Very impressive, amazing progress for a part time project ! 

Thats interesting that iOS supports M-JPEG, had not heard that before. 

Per M-JPGEG in wikipedia app ... They have WKWebView webview in iOS8 and above 
no? so in theory could run the JS engine against the same subset of iOS devices 
at similar performance as a stop gap there. But of course the native player 
would be ideal ;)

—michael


> On Jun 11, 2015, at 7:05 PM, Fabrice Florin  wrote:
> 
> Nicely done, Brion!
> 
> We’re very grateful for all the mobile multimedia work you’ve been doing on 
> your ‘spare time’ …
> 
> Much appreciated,
> 
> 
> Fabrice
> 
> 
>> On Jun 11, 2015, at 7:34 AM, Brion Vibber > > wrote:
>> 
>> I've been passing the last few days feverishly working on audio/video stuff, 
>> cause it's been driving me nuts that it's not quite in working shape.
>> 
>> TL;DR: Major fixes in the works for Android, Safari (iOS and Mac), and 
>> IE/Edge (Windows). Need testers and patch reviewers.
>> 
>> 
>> == ogv.js for Safari/IE/Edge ==
>> 
>> In recent versions of Safari, Internet Explorer, and Microsoft's upcoming 
>> Edge browser, there's still no default Ogg or WebM support but JavaScript 
>> has gotten fast enough to run an Ogg Theora/Vorbis decoder with CPU to spare 
>> for drawing and outputting sound in real time.
>> 
>> The ogv.js decoder/player has been one of my fun projects for some time, and 
>> I think I'm finally happy with my TimedMediaHandler/MwEmbedPlayer 
>> integration patch  for the 
>> desktop MediaWiki interface.
>> 
>> I'll want to update it to work with Video.js later, but I'd love to get this 
>> version reviewed and deployed in the meantime.
>> 
>> Please head over to https://ogvjs-testing.wmflabs.org/ 
>>  in Safari 6.1+ or IE 10+ (or 'Project 
>> Spartan' on Windows 10 preview) and try it out! Particularly interested in 
>> cases where it doesn't work or messes up.
>> 
>> 
>> == Non-JavaScript fallback for iOS ==
>> 
>> I've found that Safari on iOS supports QuickTime movies with Motion-JPEG 
>> video and mu-law PCM audio . 
>> JPEG and PCM are, as it happens, old and not so much patented. \o/
>> 
>> As such this should work as a fallback for basic audio and video on older 
>> iPhones and iPads that can't run ogv.js well, or in web views in apps that 
>> use Apple's older web embedding APIs where JavaScript is slow (for example, 
>> Chrome for iOS).
>> 
>> However these get really bad compression ratios, so to keep bandwidth down 
>> similar to the 360p Ogg and WebM versions I had to reduce quality and 
>> resolution significantly. Hold an iPhone at arm's length and it's maybe ok, 
>> but zoom full-screen on your iPad and you'll hate the giant blurry pixels!
>> 
>> This should also provide a working basic audio/video experience in our 
>> Wikipedia iOS app, until such time as we integrate Ogg or WebM decoding 
>> natively into the app.
>> 
>> Note that it seems tricky to bulk-run new transcodes on old files with 
>> TimedMediaHandler. I assume there's a convenient way to do it that I just 
>> haven't found in the extension maint scripts...
>> 
>> 
>> == In progress: mobile video fixes ==
>> 
>> Audio has worked on Android for a while -- the .ogg files show up in native 
>>  elements and Just Work.
>> 
>> But video has been often broken, with TimedMediaHandler's "popup transforms" 
>> reducing most video embeds into a thumbnail and a link to the original file 
>> -- which might play if WebM (not if Ogg Theora) but it might also be a 1080p 
>> original which you don't want to pull down on 3G! And neither audio nor 
>> video has worked on iOS.
>> 
>> This patch  adds a simple mobile 
>> target for TMH, which fixes the popup transforms to look better and actually 
>> work by loading up an embedded-size player with the appropriately playable 
>> transcodes (WebM, Ogg, and the MJPEG last-ditch fallback).
>> 
>> ogv.js is used if available and necessary, for instance in iOS Safari when 
>> the CPU is fast enough. (Known to work only on 64-bit models.)
>> 
>> 
>> == Future: codec.js and WebM and OGVKit ==
>> 
>> For the future, I'm also working on extending ogv.js to support WebM 
>>  
>> for better quality (especially in high-motion scenes) -- once that 
>> stabilizes I'll rename the combined package codec.js. Performance of WebM is 
>> not yet good enough to deploy, and some features like seeking are still 
>> missing, but breaking out the codec modules means I can develop the codecs 
>> in parallel and keep the high-level player logic in common.
>> 
>> Browser infrastructure improvements like SIMD, threading, and more GPU 
>> access should continue to make WebM decoding faster in the future as well.
>>  
>> 
>> I'd also like to finish up my OGVKi

Re: [WikimediaMobile] iOS Beta 4.1.5 available for testing

[Toby Negrin]

Hi Corey -- what's the version #? 4.1.5(143)?

On Thu, Jun 11, 2015 at 9:11 AM, Corey Floyd  wrote:

> Hey everyone, we have a new iOS release in the hopper. If you are part of
> the beta channel, you can download the new version using Apple's TestFlight.
>
> If you would like to be in the beta, just email me, Brian or Monte to get
> access.
>
> Here are the highlights of this release:
>
> * Lead images now load faster and fade in smoothly
> * More article layout improvements
> * More reliable loading of articles from the cache
> * Added a link to the FAQ in Settings > More
> * Fix crash on startup for some device languages (Like Norwegian)
> * Several other crash fixes
>
> Also of note: this our last "scheduled" iOS 6 release. After release, this
> version will be available for download indefinitely for our users who are
> unable to upgrade to iOS 7.
>
> For users who are running  iOS 7 or higher, this will allow us to support
> more great features in future versions of the iOS application.
>
> Thanks for testing and please send us any feedback you may have!
>
> --
> Corey Floyd
> Software Engineer
> Mobile Apps / iOS
> Wikimedia Foundation
>
> ___
> Mobile-l mailing list
> Mobile-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>
>
___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

Re: [WikimediaMobile] iOS Beta 4.1.5 available for testing

[Legoktm]

Hi,

On 06/11/2015 09:11 AM, Corey Floyd wrote:
> Thanks for testing and please send us any feedback you may have!

The language selector is now more prominent which is nice, however I
can't figure out how to get to the history view. Was that removed?

-- Legoktm

___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

Re: [WikimediaMobile] iOS Beta 4.1.5 available for testing

[Corey Floyd]

Tony - yes version is 4.1.5(143)

On Thu, Jun 11, 2015 at 1:05 PM, Toby Negrin  wrote:

> Hi Corey -- what's the version #? 4.1.5(143)?
>
> On Thu, Jun 11, 2015 at 9:11 AM, Corey Floyd  wrote:
>
>> Hey everyone, we have a new iOS release in the hopper. If you are part of
>> the beta channel, you can download the new version using Apple's TestFlight.
>>
>> If you would like to be in the beta, just email me, Brian or Monte to get
>> access.
>>
>> Here are the highlights of this release:
>>
>> * Lead images now load faster and fade in smoothly
>> * More article layout improvements
>> * More reliable loading of articles from the cache
>> * Added a link to the FAQ in Settings > More
>> * Fix crash on startup for some device languages (Like Norwegian)
>> * Several other crash fixes
>>
>> Also of note: this our last "scheduled" iOS 6 release. After release,
>> this version will be available for download indefinitely for our users
>> who are unable to upgrade to iOS 7.
>>
>> For users who are running  iOS 7 or higher, this will allow us to
>> support more great features in future versions of the iOS application.
>>
>> Thanks for testing and please send us any feedback you may have!
>>
>> --
>> Corey Floyd
>> Software Engineer
>> Mobile Apps / iOS
>> Wikimedia Foundation
>>
>> ___
>> Mobile-l mailing list
>> Mobile-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>
>>
>


-- 
Corey Floyd
Software Engineer
Mobile Apps / iOS
Wikimedia Foundation
___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

Re: [WikimediaMobile] iOS Beta 4.1.5 available for testing

[Corey Floyd]

And thats why we beta test… :)

We actually omitted the commit patch that puts that back in place. Thanks
for finding that, expect a new beta to be distributed soon.

On Thu, Jun 11, 2015 at 1:11 PM, Legoktm 
wrote:

> Hi,
>
> On 06/11/2015 09:11 AM, Corey Floyd wrote:
> > Thanks for testing and please send us any feedback you may have!
>
> The language selector is now more prominent which is nice, however I
> can't figure out how to get to the history view. Was that removed?
>
> -- Legoktm
>
> ___
> Mobile-l mailing list
> Mobile-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>



-- 
Corey Floyd
Software Engineer
Mobile Apps / iOS
Wikimedia Foundation
___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

Re: [WikimediaMobile] iOS Beta 4.1.5 available for testing

[Corey Floyd]

Just posted a new version: 4.1.5 (144) with the fix in place. Thanks again
for testing!

On Thu, Jun 11, 2015 at 1:45 PM, Corey Floyd  wrote:

> And thats why we beta test… :)
>
> We actually omitted the commit patch that puts that back in place. Thanks
> for finding that, expect a new beta to be distributed soon.
>
> On Thu, Jun 11, 2015 at 1:11 PM, Legoktm 
> wrote:
>
>> Hi,
>>
>> On 06/11/2015 09:11 AM, Corey Floyd wrote:
>> > Thanks for testing and please send us any feedback you may have!
>>
>> The language selector is now more prominent which is nice, however I
>> can't figure out how to get to the history view. Was that removed?
>>
>> -- Legoktm
>>
>> ___
>> Mobile-l mailing list
>> Mobile-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>>
>
>
>
> --
> Corey Floyd
> Software Engineer
> Mobile Apps / iOS
> Wikimedia Foundation
>



-- 
Corey Floyd
Software Engineer
Mobile Apps / iOS
Wikimedia Foundation
___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

[WikimediaMobile] [Apps] Android Wikipedia beta app release (2.0.103-beta-2015-06-11)

[Stephen Niedzielski]

  The Android team[1] is proud to announce a new Wikipedia Android app beta
release, v2.0.103-beta-2015-06-11[2]. In summary, this revision contains
the following enhancements[3]:

  * Enable link previews for even more users and improve design.
  * Improve page caching. Content is cached when possible but is always the
latest available.
  * New! Enable "Search Wikipedia" when sharing text from other apps.
  * Fix text direction for mixed left-to-right and right-to-left languages.
  * Replace Chinese language options with Simplified and Traditional
dialects.
  * Fix thumbnails in saved pages.
  * New! Use system language option.
  * Numerous translation updates.
  * Many minor fixes & improvements.

  [1] https://www.mediawiki.org/wiki/Wikimedia_Apps/Team#Android_App
  [2] Rolling out at
https://play.google.com/store/apps/details?id=org.wikipedia.beta
  [3] A complete list of changes is available   at
http://git.wikimedia.org/compare/apps%2Fandroid%2Fwikipedia/beta%2F2.0.102-beta-2015-05-14..beta%2F2.0.103-beta-2015-06-11
  [4]
https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/Wikipedia_Android_app_hacking

  Wanna do Android? Read our getting started guide[4]. We can't wait for
your contributions!
___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

Re: [WikimediaMobile] [reading-wmf] A plan for the "new" mobile view service

[S Page]

Awesome. Dmitry I shall make you explain "How I wrote a new RESTBase
service" :)

Will this replace or subsume
http://www.mediawiki.org/wiki/Extension:TextExtracts ? Will clients be able
to request first paragraph, first 3 sentences, etc.?

> In the case of entities existing within a paragraph, we can decide (with a
> little help from Design) whether it's important to keep them inline with
> the text, or strip and move them outside of the paragraph.
>
Will clients be able to request different kinds of stripping? It seems
really hard. If you look at the Vincent van Gogh article, its opening
sentence is

*Vincent Willem van Gogh* (Dutch: [ˈvɪnsɛnt ˈʋɪləm vɑn ˈɣɔx]
 (
 listen

);[note 1]  30
March 1853 – 29 July 1890) was a major Post-Impressionist
 painter.

and it seems every client shows this differently:

Google search results snippet displays

Vincent Willem van Gogh (Dutch: [ˈvɪnsɛnt ˈʋɪləm vɑn ˈɣɔx] ( listen); 30
March 1853 – 29 July 1890) was a major ...

Clearly " (listen)" shouldn't be there. Meanwhile Wikipedia Beta Android
app and Google's Knowledge graph box remove everything in parentheses
(how?) and show two sentences:
Vincent Willem van Gogh was a major Post-Impressionist painter. He was a
Dutch artist whose work had a far-reaching influence on 20th-century art.
Wikipedia 

But the Wikipedia Beta Android app's Share as image gives me:
[image: Displaying Vincent_van_Gogh.jpg]
(I filed https://phabricator.wikimedia.org/T102208 ).

It looks like the mobile view service
http://appservice.wmflabs.org/en.wikipedia.org/v1/mobile/app/page/lite/Vincent_van_Gogh
also renders the full HTML, including the "listen" speaker icon.

There's no single correct form for this snippet, it can't be decomposed
into separate bits of JSON, and the pronunciation isn't cleanly nested in
HTML for clients to easily remove the right parts of it. The mobile view
service could have an ill-defined "Do the right thing" API, or implement a
lot of named transform styles, or have some kind of domain-specific
language 8-), or always returns structured Parsoid HTML that clients strip,
or ??

Cheers,

- - - - originals to end - - - - -

On Jun 11, 2015 10:05 AM, "Dmitry Brant"  wrote:

> Yes, we should definitely do both, keeping in mind that the JSON-only
> service will be much more important for apps in the long run.
> The part that worries me a little bit is not knowing when exactly these
> services can be deployed to production at full scale. Since so many of our
> brainstorming ideas for Q1 and beyond are dependent on these services, we
> should have a concrete time frame for this.
>
> The JSON service basically already exists[1] (in its infancy), and
> experimenting with changes to the output JSON structure is absurdly easy.
> I would suggest that we take an inventory[2] of all the non-text entities
> that one might find in articles (infoboxes, tables, references, images,
> math formulas, etc), and update the service to structure them as JSON. Then
> we'll be free to decide how we want to present these entities natively in
> the apps.
>
> In the case of entities existing within a paragraph, we can decide (with a
> little help from Design) whether it's important to keep them inline with
> the text, or strip and move them outside of the paragraph.  For entities
> that are important to keep inline, we can still strip them out and
> restructure them as JSON, but also replace the inline occurrence with a
> syntax marker that the apps will recognize, and decide how to handle
> natively.
>
> Whether to preserve HTML formatting might also be a question for design /
> UX research. However, at least on Android, the native TextView does support
> some limited HTML tags, and we can do additional formatting with Spans if
> necessary.
>
>
> [1]
> http://appservice.wmflabs.org/en.wikipedia.org/v1/mobile/app/page/lite/Wombat
> [2] https://etherpad.wikimedia.org/p/json-content-service-structure
>
>
> On Thu, Jun 11, 2015 at 11:28 AM, Corey Floyd 
> wrote:
>
>>
>> Mostly apps have been talking about this, but I think it would be good to
>> get web folks involved as well.
>>
>> We have a lot of ideas, and this is at the top of the list for things we
>> need to accomplish potential goals for the quarter. It also seems there
>> are at least 2 ideas for how we should do this:
>>
>> 1. Deploy a service backed by Parsed that delivers better marked up HTML
>> than mobile View.
>> 2. Deploy a service that converts HTML to JSON and delivers that instead.
>>
>> My suggestion would be to do both… deliver better HTML first (this should
>> be "easier"), then build another service upon that to serve JSON.
>>
>> The JSON 

Re: [WikimediaMobile] [reading-wmf] A plan for the "new" mobile view service

[Gergo Tisza]

On Thu, Jun 11, 2015 at 6:14 PM, S Page  wrote:

> In the case of entities existing within a paragraph, we can decide (with a
>> little help from Design) whether it's important to keep them inline with
>> the text, or strip and move them outside of the paragraph.
>>
> Will clients be able to request different kinds of stripping? It seems
> really hard.
>
It's even harder if you want it to work outside the English Wikipedia -
different wikis might use different templates which generate slightly
different HTML.

The skinnable content snippets proposal from the brainstorming could be
abused to handle this:
** Explore the possibility of wikitext markup for handling over control
over sections of content to the skin (like
https://phabricator.wikimedia.org/T25796 but bigger / more generic; Winter
had some great design ideas that could be implemented with such a feature)

Editors would write something like this in the source code:
'''Vincent Willem van Gogh''' {{#snippet|role=pronunciation|IPA=ˈvɪnsɛnt
ˈʋɪləm vɑn ˈɣɔx}}{{#snippet|role=birth and death|birth=30 March
1853|death=29 July 1890}} was a major [[Post-Impressionist]] painter.
and the snippets would turn into different templates on different
devices/skins, in some cases not producing wikicode output at all but
instead pushing the information to a side channel.
___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

Re: [WikimediaMobile] [reading-wmf] A plan for the "new" mobile view service

[Corey Floyd]

You both are pointing out the real challenge here: The inline content.

The good news is that we don't have to worry about the wiki text or
templates. Parsoid already has dealt with that and is providing additional
markup. This makes it possible to do a 2 way conversion between
HTML<->Wikitext for VE. The goal is to build upon that work to the same
thing with "ParsoidHTML"->JSON.

The hope is that if the markup provided by Parsoid is enough to allow VE to
do the 2 way conversion, there should be enough information for us do at
least a 1 way conversion into JSON without loosing data.

Beyond using the markup to translate the HTML into JSON, we also have to
figure out how to actually represent that in JSON. Especially for inline
entities. Also, we have questions about wether we should still use a subset
of HTML to describe the text styles (like bold, italics, etc…) The
definition of the article JSON spec may be just as challenging as the
actual engineering work.

On Thu, Jun 11, 2015 at 9:51 PM, Gergo Tisza  wrote:

> On Thu, Jun 11, 2015 at 6:14 PM, S Page  wrote:
>
>> In the case of entities existing within a paragraph, we can decide (with
>>> a little help from Design) whether it's important to keep them inline with
>>> the text, or strip and move them outside of the paragraph.
>>>
>> Will clients be able to request different kinds of stripping? It seems
>> really hard.
>>
> It's even harder if you want it to work outside the English Wikipedia -
> different wikis might use different templates which generate slightly
> different HTML.
>
> The skinnable content snippets proposal from the brainstorming could be
> abused to handle this:
> ** Explore the possibility of wikitext markup for handling over control
> over sections of content to the skin (like
> https://phabricator.wikimedia.org/T25796 but bigger / more generic;
> Winter had some great design ideas that could be implemented with such a
> feature)
>
> Editors would write something like this in the source code:
> '''Vincent Willem van Gogh''' {{#snippet|role=pronunciation|IPA=ˈvɪnsɛnt
> ˈʋɪləm vɑn ˈɣɔx}}{{#snippet|role=birth and death|birth=30 March
> 1853|death=29 July 1890}} was a major [[Post-Impressionist]] painter.
> and the snippets would turn into different templates on different
> devices/skins, in some cases not producing wikicode output at all but
> instead pushing the information to a side channel.
>
> ___
> Mobile-l mailing list
> Mobile-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/mobile-l
>
>


-- 
Corey Floyd
Software Engineer
Mobile Apps / iOS
Wikimedia Foundation
___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l