[+ Ori] Joel, Ori looked into this now. There was a problem with EL in labs which affected logging events from Beta. Ori has fixed the issue, and the fix is waiting approval from ops. Let's touch-base tomorrow to see if we see events.
Leila On Thu, Nov 13, 2014 at 1:30 PM, Nuria Ruiz <nu...@wikimedia.org> wrote: > Joel: > > I see, I was hoping to set aside the beta issues but if you are not > deploying to prod any time soon I guess we will need to troubleshoot there. > By the looks of it EL has not worked in beta since august, but, as I said > before, I know very little about how beta is put together. > > I have filed a bug to regarding the beta issue: > https://bugzilla.wikimedia.org/show_bug.cgi?id=73388 > > > > > > > On Thu, Nov 13, 2014 at 12:52 PM, Joel Sahleen <jsahl...@wikimedia.org> > wrote: > >> Hi Nuria, >> >> >Please let me know if there is any way I can help out or if there is >> anything you need from our end. >> When you have deployed your newest code to production, let's check >> whether events appear on the production stream. Let us know when deployment >> is done and you think your code should be logging. >> >> >> Our code is not scheduled to be released to production until January. >> Getting the metrics is partly to help us ensure and promote that release. >> We will keep you informed as our plans progress, but hopefully we can >> figure out what the issue is in beta soon. >> >> To confirm: You have seen proper logging from your events in vagrant, >> right? >> >> >> The output I am seeing with vagrant is what I pasted to this thread >> earlier. It does not contain the url-encoded section or the user agent >> information as we discussed before. I think that is an issue with my dev >> environment, however, and not a problem with the code. The same code >> appears to be sending valid events in beta. The http request I sent to your >> email earlier is what we are seeing there. It seems to include all the >> information you said it should include. >> >> If you want to debug what is happening in beta yourself, an easy way I >> found to do that is: >> >> >> 1. Go to our Content Translation translation view >> >> <http://en.wikipedia.beta.wmflabs.org/wiki/Special:ContentTranslation?page=Han+Feizi&from=es&to=ca&targettitle=Han+Feizi> >> page >> in beta (you will need to create an account and sign in) >> 2. Open chrome dev tools, >> 3. Click the add translation links that appear in the middle column >> to add a few machine translated paragraphs to the editor >> 4. Click on the publish button in the header to publish the >> translation to your user namespace (triggers EL event) >> 5. Look at the network pane in chrome dev tools and find the entry >> with the event logging url (it should be near the bottom). >> 6. Click on the entry to see all the request and response information. >> >> >> You probably already know all this, but I thought I would pass it along >> just in case it helps. >> >> Di you setup a sampling rate or code is logging 1 to 1? >> >> >> No sample rate. Just logging 1 to 1. >> >> On our end we will work to troubleshoot the beta EL infrastructure, I am >> not familiar with it and neither is anyone on our team but we will ask >> around. >> >> >> Yeah, Dan said you all kind of inherited EL so that’s totally >> understandable. We appreciate you looking into this for us. Let us know how >> else we can help. >> >> Joel >> >> >> >> >> >> On Thu, Nov 13, 2014 at 8:45 AM, Joel Sahleen <jsahl...@wikimedia.org> >> wrote: >> >>> Hi Nuria, >>> >>> Thank you so much for your help on this. Please let me know if there is >>> any way I can help out or if there is anything you need from our end. >>> >>> Joel >>> >>> Joel Sahleen, Software Engineer >>> Language Engineering >>> Wikimedia Foundation >>> jsahl...@wikimedia.org >>> >>> >>> >>> >>> On Nov 13, 2014, at 9:42 AM, Nuria Ruiz <nu...@wikimedia.org> wrote: >>> >>> Hello, >>> >>> Taking last statement back, asked Yuvi and beta does have a varnish >>> instance so the flow of EL events "should" be the same one that production. >>> >>> Now I looked on deployment-eventlogging02, which is the EL machine for >>> labs and the last events I see there are from Aug 22. >>> >>> So no events have come in as of late, which could point to an issue on >>> the setup. I will look into it some more. >>> >>> Thanks, >>> >>> Nuria >>> >>> On Wed, Nov 12, 2014 at 10:40 AM, Nuria Ruiz <nu...@wikimedia.org> >>> wrote: >>> >>>> To keep archives happy: Beta setup post events to >>>> http://bits.beta.wmflabs.org/event.gif >>>> <http://bits.beta.wmflabs.org/event.gif?foo=bar> that, while it does >>>> not look to be varnish, has some kind of listener that post those events to >>>> beta event logging database. >>>> >>>> On Wed, Nov 12, 2014 at 9:37 AM, Joel Sahleen <jsahl...@wikimedia.org> >>>> wrote: >>>> >>>>> Niklas, >>>>> >>>>> Can you answer this question from Nuria? >>>>> >>>>> jsahleen: does beta have its own varnish instance? where are you >>>>> posting your events in beta? can you send teh url? >>>>> >>>>> Also would it be possible to document the steps you used when testing >>>>> EL on beta so that others can reproduce them? >>>>> >>>>> Thanks, >>>>> >>>>> Joel >>>>> >>>>> Joel Sahleen, Software Engineer >>>>> Language Engineering >>>>> Wikimedia Foundation >>>>> jsahl...@wikimedia.org >>>>> >>>>> >>>>> >>>>> >>>>> On Nov 12, 2014, at 4:28 AM, Joel Sahleen <jsahl...@wikimedia.org> >>>>> wrote: >>>>> >>>>> (Moving this discussion to analytics@ and localization-team@ based on >>>>> Nuria’s suggestion below.) >>>>> >>>>> Hi Leila, >>>>> >>>>> The output I posted in the message is the only output I am seeing. I >>>>> do not see the URL-encoded section or the validation section. I think >>>>> there >>>>> may be something wrong with my testing setup. >>>>> >>>>> Niklas Laxstöm has checked what is happening with our event logging in >>>>> beta and he confirmed that we are sending events and the events are valid. >>>>> The issue seems to be that we are logging events to the beta event logging >>>>> db while what we checked earlier was the production event logging db. >>>>> >>>>> Can you (or anyone who is available) check the event logging db in >>>>> beta to see if the table has been created and has data? The schema name >>>>> again is ContentTranslation. If you don’t find anything, let us know and >>>>> we >>>>> will do some more investigation. >>>>> >>>>> If there is data in the beta db the next step would be to follow with >>>>> Dan’s >>>>> instructions >>>>> <https://wikitech.wikimedia.org/wiki/Analytics/Dashboards> to get a >>>>> dashboard set up on limn1. I believe that most of Dan’s instructions need >>>>> to be handled by someone on the analytics team, but let me know if there >>>>> is >>>>> anything I can help with. >>>>> >>>>> Thanks again for your help! >>>>> >>>>> Joel >>>>> >>>>> Joel Sahleen, Software Engineer >>>>> Language Engineering >>>>> Wikimedia Foundation >>>>> jsahl...@wikimedia.org >>>>> >>>>> >>>>> >>>>> >>>>> On Nov 11, 2014, at 11:47 PM, Leila Zia <le...@wikimedia.org> wrote: >>>>> >>>>> Hi Joel, >>>>> >>>>> When you log events, the output will be the URL-encoded JSON sent >>>>> by the browser, the event record (similar to what you pasted in your >>>>> email), and whether the event validates against the schema. For the sample >>>>> output you pasted earlier, or another sample output, can you let us know >>>>> if >>>>> validation section shows Valid? >>>>> >>>>> Leila >>>>> >>>>> On Mon, Nov 10, 2014 at 3:24 PM, Nuria Ruiz <nu...@wikimedia.org> >>>>> wrote: >>>>> >>>>>> Joel, >>>>>> >>>>>> For questions like these going forward you can contact analytics@ as >>>>>> you will be getting amore prompt response. Both Dan and Leila are OOTO >>>>>> the >>>>>> next couple of days. >>>>>> >>>>>> >There are configuration options for the dev server that need to be >>>>>> added. Do similar options need to be added when not using the dev server? >>>>>> No, there is no need. >>>>>> >>>>>> You would need sample rates to determine at which sampling rate you >>>>>> are logging if you are not logging all events, that is. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Nuria >>>>>> >>>>>> On Mon, Nov 10, 2014 at 2:39 PM, Dan Andreescu < >>>>>> dandree...@wikimedia.org> wrote: >>>>>> >>>>>>> Adding Nuria as she can probably help >>>>>>> >>>>>>> On Monday, November 10, 2014, Joel Sahleen <jsahl...@wikimedia.org> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Leila, >>>>>>>> >>>>>>>> I have tested our EventLogging code and it seems to be working fine >>>>>>>> with the event logging dev server. I can see the events coming through >>>>>>>> and >>>>>>>> they are valid. Here is some sample output: >>>>>>>> >>>>>>>> {"wiki": "wiki", "uuid": "e9dde14cf18552269ae81a7897f45d0c", >>>>>>>> "webHost": "localhost", "timestamp": 1415651367, "clientValidated": >>>>>>>> true, >>>>>>>> "recvFrom": "1.0.0.127.in-addr.arpa", "seqId": 2, "clientIp": >>>>>>>> "80f7683f3565e3d365740a1c8d1771ba95caaaaa", "schema": >>>>>>>> "ContentTranslation", >>>>>>>> "event": {"action": "create-translated-page", "targetLanguage": "ca", >>>>>>>> "token": "Tester", "version": 1, "contentLanguage": "es"}, "revision": >>>>>>>> 7146627} >>>>>>>> >>>>>>>> Are there additional configuration options we need to add to get EL >>>>>>>> working aside from just requiring the main extension file. There are >>>>>>>> configuration options for the dev server that need to be added. Do >>>>>>>> similar >>>>>>>> options need to be added when not using the dev server? >>>>>>>> >>>>>>>> Any help on this would be much appreciated. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Joel >>>>>>>> >>>>>>>> On Nov 7, 2014, at 3:52 PM, Joel Sahleen <jsahl...@wikimedia.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>> No problem, Dan. Enjoy your vacation! >>>>>>>> >>>>>>>> I will read through the document at the link you sent. I still need >>>>>>>> to fix our event logging code so it may be a couple days before we are >>>>>>>> ready anyway. If I have any questions I will contact Leila or Nuria. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Joel >>>>>>>> >>>>>>>> Joel Sahleen, Software Engineer >>>>>>>> Language Engineering >>>>>>>> Wikimedia Foundation >>>>>>>> jsahl...@wikimedia.org >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Nov 7, 2014, at 3:10 PM, Dan Andreescu <dandree...@wikimedia.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Joel, re: visualization, >>>>>>>> >>>>>>>> I'm going on vacation tomorrow and will be back on November 19th. >>>>>>>> If that's not too late, I can set up a limn instance then. If it's too >>>>>>>> late, that's ok, I wrote up the steps needed. Someone with access to >>>>>>>> the >>>>>>>> limn1.eqiad.wmflabs instance can perform them: >>>>>>>> https://wikitech.wikimedia.org/wiki/Analytics/Dashboards >>>>>>>> >>>>>>>> If you have the data or are generating the data in some other way, >>>>>>>> then you don't need half of that setup, you just need the part that >>>>>>>> sets up >>>>>>>> the limn dashboard which is only an hour or so of work. Sorry I'm >>>>>>>> running >>>>>>>> out the door and can't take care of that for you. >>>>>>>> >>>>>>>> Dan >>>>>>>> >>>>>>>> On Fri, Nov 7, 2014 at 7:37 AM, Joel Sahleen < >>>>>>>> jsahl...@wikimedia.org> wrote: >>>>>>>> >>>>>>>>> Thank you for the information, Pau. Very helpful. As you say, this >>>>>>>>> does not change our current plans or hold us up in any way. I was just >>>>>>>>> wasn’t clear about the relationship between the "high priorities" and >>>>>>>>> "other metrics” sections. Knowing these came from different people at >>>>>>>>> different times clarifies things a lot. >>>>>>>>> Joel >>>>>>>>> >>>>>>>>> On Nov 7, 2014, at 3:44 AM, Pau Giner <pgi...@wikimedia.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> @Pau, @Amir There is a section called High priorities for product >>>>>>>>>> management >>>>>>>>>> <https://www.mediawiki.org/wiki/Content_translation/analytics#High_priorities_for_product_management> >>>>>>>>>> on >>>>>>>>>> the Content translation analytics page. Did these priorities come >>>>>>>>>> from >>>>>>>>>> outside the team or does this just represent our own internal view >>>>>>>>>> of the >>>>>>>>>> high priorities? >>>>>>>>> >>>>>>>>> >>>>>>>>> Here is the story of that page as I'm aware of it: >>>>>>>>> >>>>>>>>> In September 2013, I was in a meeting with the analytics team in >>>>>>>>> SF presenting an initial proposal for metrics >>>>>>>>> <https://docs.google.com/a/wikimedia.org/presentation/d/1V1XLV7jUcAtco5ZC49SNTt3VecH7hARZ6vqbSFGnOYc/edit?usp=sharing>. >>>>>>>>> On that meeting, Dario recommended to create hierarchy of metrics >>>>>>>>> based on >>>>>>>>> the project goals. I created such image and a description for those >>>>>>>>> metrics >>>>>>>>> (the image is on top of our analytics page and the metrics are >>>>>>>>> described in >>>>>>>>> what it now the "Other metrics for created articles" section. >>>>>>>>> >>>>>>>>> In a meeting between Amir and Howie, they captured which should be >>>>>>>>> the most important metrics from the product perspective in the "High >>>>>>>>> priorities for product management". If I recalled correctly, as an >>>>>>>>> outcome >>>>>>>>> of later meetings between Howie and Amir, Howie was happy focusing on >>>>>>>>> articles published as a single (initial?) metric for success. Amir can >>>>>>>>> provide more details since I was not on those meetings. >>>>>>>>> >>>>>>>>> In short: The analytics page >>>>>>>>> <https://www.mediawiki.org/wiki/Content_translation/analytics> >>>>>>>>> has pieces contributed by different people during the last year, and >>>>>>>>> although there are many ideas to organise and detail, measuring the >>>>>>>>> number >>>>>>>>> of published articles seems to be the solid candidate to get started >>>>>>>>> with, >>>>>>>>> learn from the value we get from it and polish the rest of our >>>>>>>>> goal-to-signal >>>>>>>>> process <http://www.rodden.org/kerry/heart/> for detecting better >>>>>>>>> metrics. >>>>>>>>> >>>>>>>>> >>>>>>>>> Pau >>>>>>>>> >>>>>>>>> On Fri, Nov 7, 2014 at 1:57 AM, Joel Sahleen < >>>>>>>>> jsahl...@wikimedia.org> wrote: >>>>>>>>> >>>>>>>>>> Hi All, >>>>>>>>>> >>>>>>>>>> I have been reviewing our requirements for Content translation >>>>>>>>>> analytics >>>>>>>>>> <https://www.mediawiki.org/wiki/Content_translation/analytics> and >>>>>>>>>> I have a few questions/requests. I am sending them to the language >>>>>>>>>> team >>>>>>>>>> list and Leila and Dan in the hopes of getting some more clarity. I >>>>>>>>>> will >>>>>>>>>> add the same content to the Trello card. >>>>>>>>>> >>>>>>>>>> In the weekly team meeting earlier today we agreed that the first >>>>>>>>>> metric we want to collect data for is the number of articles created >>>>>>>>>> in >>>>>>>>>> each language over time. This is something has Amir has already set >>>>>>>>>> up our >>>>>>>>>> current Event Logging >>>>>>>>>> <https://git.wikimedia.org/blob/mediawiki/extensions/ContentTranslation/89b6284f06b4419ddec6dcccee0eed500f267100/modules/eventlogging/ext.cx.eventlogging.js> >>>>>>>>>> to >>>>>>>>>> track. Now that Kartik has enabled EL in beta, that part should be >>>>>>>>>> done. >>>>>>>>>> Since we are only barely turning it on, there will be very little >>>>>>>>>> data >>>>>>>>>> until people create more articles using CX. However, we should be >>>>>>>>>> set up to >>>>>>>>>> collect any new data that comes in. >>>>>>>>>> >>>>>>>>>> @Leila, can you verify that the db table now exists for the >>>>>>>>>> ContentTranslation >>>>>>>>>> schema >>>>>>>>>> <https://meta.wikimedia.org/wiki/Schema:ContentTranslation>? If >>>>>>>>>> it doesn’t, can you point us to right people we need to work with to >>>>>>>>>> troubleshoot the issue? Also you mentioned in our meeting that >>>>>>>>>> personal >>>>>>>>>> data may soon be purged after 90 days as part of a new privacy >>>>>>>>>> policy. >>>>>>>>>> Could you explain that a bit more or point us to more information? >>>>>>>>>> If this >>>>>>>>>> is the case, it may affect some of the metrics we would like to >>>>>>>>>> collect in >>>>>>>>>> the future. >>>>>>>>>> >>>>>>>>>> @Dan, what do we need to do next in order to set up a very simple >>>>>>>>>> visualization that would show the number of articles created per >>>>>>>>>> week by >>>>>>>>>> language. Pau has an image of what he would like on the Trello >>>>>>>>>> card >>>>>>>>>> <https://trello.com/c/vQm0hlkt/18-content-translation-analytics>. >>>>>>>>>> You mentioned something about being able to host a dashboard for us >>>>>>>>>> on one >>>>>>>>>> of the Limn servers you already have set up. >>>>>>>>>> >>>>>>>>>> @Santhosh, I believe you said earlier you have a script you use >>>>>>>>>> to export the data for the ULS analytics. If so can you share that >>>>>>>>>> please >>>>>>>>>> in case we need a similar script for CX so I don’t have to write a >>>>>>>>>> new >>>>>>>>>> script from scratch? >>>>>>>>>> >>>>>>>>>> @Pau, @Amir There is a section called High priorities for >>>>>>>>>> product management >>>>>>>>>> <https://www.mediawiki.org/wiki/Content_translation/analytics#High_priorities_for_product_management> >>>>>>>>>> on >>>>>>>>>> the Content translation analytics page. Did these priorities come >>>>>>>>>> from >>>>>>>>>> outside the team or does this just represent our own internal view >>>>>>>>>> of the >>>>>>>>>> high priorities? If the latter, have these priorities been >>>>>>>>>> reviewed by anyone outside the team? I think we are safe to proceed >>>>>>>>>> with >>>>>>>>>> our current plan, but it would be good to have product sign off on >>>>>>>>>> things >>>>>>>>>> more generally. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Joel >>>>>>>>>> >>>>>>>>>> Joel Sahleen, Software Engineer >>>>>>>>>> Language Engineering >>>>>>>>>> Wikimedia Foundation >>>>>>>>>> jsahl...@wikimedia.org >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Localisation-team mailing list >>>>>>>>>> localisation-t...@lists.wikimedia.org >>>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/localisation-team >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Pau Giner >>>>>>>>> Interaction Designer >>>>>>>>> Wikimedia Foundation >>>>>>>>> _______________________________________________ >>>>>>>>> Localisation-team mailing list >>>>>>>>> localisation-t...@lists.wikimedia.org >>>>>>>>> https://lists.wikimedia.org/mailman/listinfo/localisation-team >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Analytics mailing list >>>>> Analytics@lists.wikimedia.org >>>>> https://lists.wikimedia.org/mailman/listinfo/analytics >>>>> >>>>> >>>> >>> _______________________________________________ >>> Localisation-team mailing list >>> localisation-t...@lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/localisation-team >>> >>> >>> >>> _______________________________________________ >>> Analytics mailing list >>> Analytics@lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/analytics >>> >>> >> _______________________________________________ >> Localisation-team mailing list >> localisation-t...@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/localisation-team >> >> >> >> _______________________________________________ >> Analytics mailing list >> Analytics@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/analytics >> >> > > _______________________________________________ > Analytics mailing list > Analytics@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/analytics > >
_______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
