Team: Besides the ability of testing in beta labs and the monitoring that ori highlited the incoming raw stream of events is available in 1003/1002 on port 8600.
>From 1002 or 1003 you can run: zsub vanadium.eqiad.wmnet:8600 and see the incoming stream. I am not sure that something beyond that is needed, please check it out and let us know. Thanks, Nuria On Thu, Dec 11, 2014 at 4:44 PM, Dario Taraborelli < dtarabore...@wikimedia.org> wrote: > thanks for the quick turnaround. > > On Dec 11, 2014, at 4:28 PM, Ori Livneh <o...@wikimedia.org> wrote: > > There's this graph: > https://graphite.wikimedia.org/render/?width=586&height=308&_salt=1418343627.977&from=-1weeks&target=movingMedian(diffSeries(eventlogging.overall.raw.rate%2Ceventlogging.overall.valid.rate)%2C20) > > The key is > 'diffSeries(eventlogging.overall.raw.rate,eventlogging.overall.valid.rate)', > which gets you the rate of invalid events per second. > > It is not broken down by schema, though. > > > this is great for monitoring, for QA purposes we really need the raw data > > We can't write invalid events to a database -- at least not the same way > we write well-formed events. The table schema is derived from the event > schema, so an invalid event would violate the constraints of the table as > well. > > > rrright > > It's possible (and easy) to set something up that watches invalid events > in real-time and does something with them. The question is: what? E-mail an > alert? Produce a daily report? Generate a graph? > > If you describe how you’d like to consume the data, I can try to hash out > an an implementation with Nuria and Christian. > > > a JSON log like all-events.log but sync’ed from vanadium more frequently > would do the job for me. It can also be truncated as we probably only need > a relatively short time window and the complete data is captured in > all-events anyway. > > D > > _______________________________________________ > Analytics mailing list > Analytics@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/analytics > >
_______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
