Thanks!
yes this is not exactly what we expected :(
I guess it was too good to be true: reduce latency and improve quality
at the same time :)
On our side I'd say that perf was the main issue, Erik added a cache at
the backend-end level which seems to have a good impact.
Morelike queries are still routed to the new datacenter in dallas to
reduce stress on eqiad. We could maybe try to reroute them to eqiad and
see if caching is sufficient?
If it's the case I'd say that we don't need to run any A/B test.
Random questions:
Is it possible to analyze the correlation between the chosen article and
the presence of an image?
Rescoring options are slightly different for enwiki, is it possible to
have the detail for e.g. enwiki/frwiki/dewiki?
If it does not require huge effort on your side I'd say that you could
run another A/B test by disabling boostLinks, you just have to add
cirrusBoostLinks=no to api URL.
Thank you
Le 26/02/2016 00:33, Erik Bernhardson a écrit :
ouch, that is not at all the result we were hoping for. Just goes to
show why we have to test these things and not just take a few examples
that perform badly in one set and look to do a better job with some
different options. Thanks for putting this together!
On Thu, Feb 25, 2016 at 3:06 PM, Dmitry Brant <dbr...@wikimedia.org
<mailto:dbr...@wikimedia.org>> wrote:
Hello all,
As mentioned previously, the current version of the Android app
contains an A/B test where it presents "read more" suggestions to
the user, based on (a) the standard "morelike" query, or (b) the
new "opening_text" query.
Here are the results from the last ~10 days of the test[0]:
- The clickthrough rate using the default morelike query is (and
has been) around 15%.
- With the new opening_text query, the clickthrough rate decreases
to about 12%:
Inline image 1
Therefore, it seems that the new query has a nontrivial negative
effect on CTR :(
We'll plan on removing this test in the next release of the app,
but we'll be happy to plug in a different or updated query, if it
will be of further use to Discovery.
[0]
https://docs.google.com/a/wikimedia.org/spreadsheets/d/1BFsrAcPgexQyNVemmJ3k3IX5rtPvJ_5vdYOyGgS5R6Y/edit?usp=sharing
(queries embedded as comments in the headers)
--
Dmitry Brant
Senior Software Engineer / Product Owner (Android)
Wikimedia Foundation
https://www.mediawiki.org/wiki/Wikimedia_mobile_engineering
_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org <mailto:Mobile-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/mobile-l
_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l
_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mobile-l
