Re: [Wikimedia-search] Gerrit Cleanup Day on Wed 23rd: Are you ready?
Hi Andre, Our plans are documented on mediawiki.org: https://www.mediawiki.org/wiki/Discovery_plans_for_gerrit_cleanup_day_2015 In terms of main point of contact for technical matters, that would be our tech lead, Erik Bernhardson. If you're looking for a contact to make product decisions, that would be me. Hope that helps. Let us know what you think of our plans. Thanks, Dan On 16 September 2015 at 08:24, Andre Klapper <aklap...@wikimedia.org> wrote: > Hi Discovery team, > > the Gerrit Cleanup Day on Wed 23rd is approaching fast - only one week > left. More info: https://phabricator.wikimedia.org/T88531 > > Do you feel prepared for the day and all team members know what to do? > > If not, what are you missing and how can we help? > > Some Gerrit queries for each team are listed under "Gerrit queries per > team/area" in https://phabricator.wikimedia.org/T88531 > Are they helpful and a good start? Or do they miss some areas (or do > you have existing Gerrit team queries to use instead or to > "integrate",e.g. for parts of MediaWiki core you might work on)? > > Also, which person will be the main team contact for the day (and > available in #wikimedia-dev on IRC) and help organize review work in > your areas, so other teams could easily reach out? > Some team plates are emptier than others so they're wondering where and > how to lend a helping hand (to find out in advance, due to timezones). > > Thanks for your help to make the Gerrit Cleanup day a success! > > andre > > -- > Andre Klapper | Wikimedia Bugwrangler > http://blogs.gnome.org/aklapper/ > > > -- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation ___ Wikimedia-search mailing list Wikimedia-search@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikimedia-search
Re: [Wikimedia-search] IRC norms
On 11 September 2015 at 13:26, Tomasz Finc <tf...@wikimedia.org> wrote: > I'd like to hear from some of our newer staff if they would have > benefited if we had this in place. Happy to draft it if we know that > it would have helped on-boarding. Agreed. I was well versed in Wikimedia IRC practices before I joined the team, so it'd be interesting to know whether people we've recently onboarded would've found it useful before we expend the effort to write it. Thanks, Dan -- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation ___ Wikimedia-search mailing list Wikimedia-search@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikimedia-search
[Wikimedia-search] Fwd: Discovery Department A/B testing an alternative to prefix search next week
Cross-posting from wikitech-l. Please reply there. -- Forwarded message -- From: Dan Garry <dga...@wikimedia.org> Date: 1 September 2015 at 20:43 Subject: Discovery Department A/B testing an alternative to prefix search next week To: Wikimedia developers <wikitec...@lists.wikimedia.org> Hi everyone, *tl;dr: Discovery Department to run A/B test <https://phabricator.wikimedia.org/T111078> comparing new search suggester to prefix search, to see if it can reduce zero results rate.* As I'm sure you're all aware, the search box at the top right of every page on desktop uses prefix search to generate its results. The main reason for this is that prefix search is incredibly fast and performant; that search box sees a lot of traffic, and it's important to keep it scalable. However, we know that there are numerous problems with prefix search. Prefix searches are prone to give you no results; if you make even a slight typo, then you won't get the result you want. And thus a complex system of manually curated redirects were born to try to alleviate this navigation issue. Wouldn't it be nice if we could work towards a solution that doesn't require the manual curation of redirects, thus freeing up Wikimedians to do other more meaningful tasks? And make search a bit better in the process, too? That's a long term goal of mine... emphasis on the long. ;-) The Q1 2015-17 (Jul - Aug 2015) goal of the Search Team in the Discovery Department is to reduce the zero results rate <https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q1_Goals#Search>. Amongst other things, we've been working to build an alternative to prefix search <https://phabricator.wikimedia.org/T105746>. Documentation on the API is pretty light right now because we're scrambling to get it up and running (but there's a task for that! <https://phabricator.wikimedia.org/T39>). An initial version of the suggestion API is now in production on enwiki and dewiki [1], but is currently not being used for anything. Our initial tests <https://phabricator.wikimedia.org/T109729> of the API show that it's incredibly promising for reducing the zero results rate. But we need more data! We're planning on running an A/B test on whether this API is better at reducing zero results. We're targeting beginning on Tuesday 8th September, for two weeks. This is documented in T111078 <https://phabricator.wikimedia.org/T111078>. A very important note here is that we currently have no way of quantitatively measuring result relevance (although we're working on it <https://phabricator.wikimedia.org/T109482>), so this test will be highly limited in scope, only measuring the zero results rate. Given the limits of this, even seeing massive success in this test is not enough to deploy this API as a full replacement of prefix search; we'd need additional data. But, that's not stopping us from gathering initial data from this test. As always, if you have any questions, let me know. Thanks, Dan [1]: The API is actually live on all wikis, but we only built the search indices for enwiki and dewiki since they're our biggest content wikis and this is an early test. Attempting to use the API on any other wiki will get you a cirrus backend error. -- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation -- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation ___ Wikimedia-search mailing list Wikimedia-search@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikimedia-search
Re: [Wikimedia-search] Measuring user user satisfaction while reducing it at the same time?
Nice catch Max. Thanks for reporting it. Do you have any suggestions for how we could alleviate this issue? Thanks, Dan On 26 August 2015 at 13:30, Max Semenik maxsem.w...@gmail.com wrote: While doing CR for https://gerrit.wikimedia.org/r/#/c/232896/3/modules/ext.wikimediaEvents.search.js I came to have serious doubts about this approach. In brief, it attempts to track user satisfaction with search results by measuring how long do people stay on pages. It does that by appending fromsearch=1 to links for 0.5% of users. However, this results in page views being uncached and thus increasing HTML load time by a factor of 4-5 and, consequentially, kicking even short pages' first paint outside of comfort zone of 1 second - and that's measured from the office, with ping of 2-3 ms to ulsfo. My concern here is that as a result we're trying to measure the very metric we're screwing with, resulting in experiment being inaccurate. Can we come up with a way of measurement that's less intrusive or alter the requirements of the experiment? -- Best regards, Max Semenik ([[User:MaxSem]]) ___ Wikimedia-search mailing list Wikimedia-search@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikimedia-search -- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation ___ Wikimedia-search mailing list Wikimedia-search@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikimedia-search
[Wikimedia-search] Fwd: Discovery Department running A/B tests for search suggestions
Cross-posting from wikitech-l. If you have any questions or comments, please post them there. Thanks, Dan -- Forwarded message -- From: Dan Garry dga...@wikimedia.org Date: 7 August 2015 at 13:19 Subject: Discovery Department running A/B tests for search suggestions To: Wikimedia developers wikitec...@lists.wikimedia.org Hello! As part of our goal to reduce the zero results rate https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q1_Goals#Search, the Discovery Department is currently running an A/B test to try different parameters for the search suggester. We're hoping that our new parameters will give users more suggestions without decreasing their quality. The reason we've chosen to tweak the suggestions is because of our recent work https://phabricator.wikimedia.org/T105202 to automatically run queries for the user if they get zero results but have a suggestion. The purpose of this A/B test is to determine whether this has significant impact towards achieving our goal or not. This is the first A/B test that the Discovery Department has run, so we're still ironing out the process. We hope to run many more A/B tests in the future. For further information on this, please review the associated Phabricator task https://phabricator.wikimedia.org/T108103. If you have any questions, I'd be happy to answer them. Thanks, Dan -- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation -- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation ___ Wikimedia-search mailing list Wikimedia-search@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikimedia-search
