[Analytics] Writing EventLogging events to database failed on 2014-11-18 for ~50 minutes between 14:14 and 15:02

2014-11-18 Thread Christian Aistleitner
Hi, the m2 master crashed today (investigation still ongoing), and caused EventLogging to not be able to write events to the database on 2014-11-18 between 14:14 and 15:02. The data for that period is not lost, but is available in backup files, waiting to get injected again into the database. Ju

Re: [Analytics] Writing EventLogging events to database failed on 2014-11-18 for ~50 minutes between 14:14 and 15:02

2014-11-18 Thread Ryan Kaldari
It looks like the same problem is happening now. No new events have been written to the log tables on analytics-store for about the past hour and a half. And it looks like the slave db stopped replicating about 6 hours ago. Ryan Kaldari On Tue, Nov 18, 2014 at 9:31 AM, Christian Aistleitner < chr

Re: [Analytics] Writing EventLogging events to database failed on 2014-11-18 for ~50 minutes between 14:14 and 15:02

2014-11-18 Thread Aaron Halfaker
Note that it appears that enwiki on analytics-slave appears to be up to date. [enwiki]> SELECT TIMEDIFF(NOW(), MAX(rc_timestamp)) FROM recentchanges; ++ | TIMEDIFF(NOW(), MAX(rc_timestamp)) | ++ | 00:00:00.00