Re: [WikimediaMobile] Results from similar articles A/B test

[David Causse]

Thanks!

yes this is not exactly what we expected :(
I guess it was too good to be true: reduce latency and improve quality 
at the same time :)
On our side I'd say that perf was the main issue, Erik added a cache at 
the backend-end level which seems to have a good impact.
Morelike queries are still routed to the new datacenter in dallas to 
reduce stress on eqiad. We could maybe try to reroute them to eqiad and 
see if caching is sufficient?

If it's the case I'd say that we don't need to run any A/B test.

Random questions:
Is it possible to analyze the correlation between the chosen article and 
the presence of an image?
Rescoring options are slightly different for enwiki, is it possible to 
have the detail for e.g. enwiki/frwiki/dewiki?


If it does not require huge effort on your side I'd say that you could 
run another A/B test by disabling boostLinks, you just have to add 
cirrusBoostLinks=no to api URL.


Thank you

Le 26/02/2016 00:33, Erik Bernhardson a écrit :
ouch,  that is not at all the result we were hoping for. Just goes to 
show why we have to test these things and not just take a few examples 
that perform badly in one set and look to do a better job with some 
different options. Thanks for putting this together!



On Thu, Feb 25, 2016 at 3:06 PM, Dmitry Brant > wrote:


Hello all,

As mentioned previously, the current version of the Android app
contains an A/B test where it presents "read more" suggestions to
the user, based on (a) the standard "morelike" query, or (b) the
new "opening_text" query.

Here are the results from the last ~10 days of the test[0]:
- The clickthrough rate using the default morelike query is (and
has been) around 15%.
- With the new opening_text query, the clickthrough rate decreases
to about 12%:

Inline image 1

Therefore, it seems that the new query has a nontrivial negative
effect on CTR :(
We'll plan on removing this test in the next release of the app,
but we'll be happy to plug in a different or updated query, if it
will be of further use to Discovery.


[0]

https://docs.google.com/a/wikimedia.org/spreadsheets/d/1BFsrAcPgexQyNVemmJ3k3IX5rtPvJ_5vdYOyGgS5R6Y/edit?usp=sharing
(queries embedded as comments in the headers)

-- 
Dmitry Brant

Senior Software Engineer / Product Owner (Android)
Wikimedia Foundation
https://www.mediawiki.org/wiki/Wikimedia_mobile_engineering


___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org 
https://lists.wikimedia.org/mailman/listinfo/mobile-l




___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l


___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

[WikimediaMobile] [offtopic] interesting read on Android app architecture

[Brian Gerstle]

And general architecture:

http://devblog.songkick.com/2016/02/25/ingredients-for-a-healthy-android-codebase/

Post links to a presentation as well. Their approach is following some best
practices I also strongly agree with, and they're using frameworks I would
probably look to use (Dagger & RxJava).



-- 
EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle
IRC: bgerstle
___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

Re: [WikimediaMobile] [reading-wmf] Usability of the mobile web site

[Jon Robson]

I've submitted patches on 2 of the 3 issues. Review welcomed.
https://gerrit.wikimedia.org/r/273542
https://gerrit.wikimedia.org/r/273540


On Thu, Feb 25, 2016 at 10:09 PM, Ori Livneh  wrote:
> So, with mobile usage creeping up to and even exceeding desktop, I figured I
> was overdue for switching over to the mobile skin, so I get to know it a bit
> better.
>
> I think it looks very attractive and modern. But it is marred by a usability
> issue so severe that it is borderline unusable to me, and that is the manner
> in which the content jumps around as the page is loading.
>
> The problem is that sections are initially loaded in an expanded state, and
> then collapsed by JavaScript code which is only executed on document ready.
> This code also pulls in interface elements that are drawn quite late. Even
> on a fast connection, a quick reader can be halfway through reading a
> paragraph when all of a sudden the content suddenly and rudely shifts to
> side or is revoked entirely.
>
> The result is very dizzying and unpleasant. As a result, I find that I have
> to train myself *not* to start reading the text in front of me until the
> page has settled down.
>
> I don't think that this is good performance. It may get something to render
> sooner than it otherwise would have, but it ends up delaying the time it
> takes for the page to settle into its fully-loaded layout, which forces the
> user to wait longer (or tolerate the electric jolt of having the content
> skip around mid-sentence).
>
> Roan filed this as https://phabricator.wikimedia.org/T126825 . I set its
> priority to "high", and I hope this e-mail gives some context as to why.
>
> ___
> reading-wmf mailing list
> reading-...@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/reading-wmf
>

___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l

Re: [WikimediaMobile] [reading-wmf] Usability of the mobile web site

[Roan Kattouw]

Excellent! I've merged both.

Also happy to hear you guys are considering leaving sections uncollapsed,
and using A/B test data to make that decision.

On Fri, Feb 26, 2016 at 5:22 PM, Jon Robson  wrote:

> I've submitted patches on 2 of the 3 issues. Review welcomed.
> https://gerrit.wikimedia.org/r/273542
> https://gerrit.wikimedia.org/r/273540
>
>
> On Thu, Feb 25, 2016 at 10:09 PM, Ori Livneh  wrote:
> > So, with mobile usage creeping up to and even exceeding desktop, I
> figured I
> > was overdue for switching over to the mobile skin, so I get to know it a
> bit
> > better.
> >
> > I think it looks very attractive and modern. But it is marred by a
> usability
> > issue so severe that it is borderline unusable to me, and that is the
> manner
> > in which the content jumps around as the page is loading.
> >
> > The problem is that sections are initially loaded in an expanded state,
> and
> > then collapsed by JavaScript code which is only executed on document
> ready.
> > This code also pulls in interface elements that are drawn quite late.
> Even
> > on a fast connection, a quick reader can be halfway through reading a
> > paragraph when all of a sudden the content suddenly and rudely shifts to
> > side or is revoked entirely.
> >
> > The result is very dizzying and unpleasant. As a result, I find that I
> have
> > to train myself *not* to start reading the text in front of me until the
> > page has settled down.
> >
> > I don't think that this is good performance. It may get something to
> render
> > sooner than it otherwise would have, but it ends up delaying the time it
> > takes for the page to settle into its fully-loaded layout, which forces
> the
> > user to wait longer (or tolerate the electric jolt of having the content
> > skip around mid-sentence).
> >
> > Roan filed this as https://phabricator.wikimedia.org/T126825 . I set its
> > priority to "high", and I hope this e-mail gives some context as to why.
> >
> > ___
> > reading-wmf mailing list
> > reading-...@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/reading-wmf
> >
>
___
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l