Re: [Analytics] Issues on Cluster

2016-02-01 Thread Andrew Otto
Woohoo! Thanks so much Joseph!!! :) On Sun, Jan 31, 2016 at 4:49 AM, Joseph Allemandou < jalleman...@wikimedia.org> wrote: > Hi All, > Everything is back to normal on the cluster, no data loss has been > incurred and jobs are up-to-date. > You can get back to your normal utilisation ! > Thanks >

Re: [Analytics] Issues on Cluster

2016-01-31 Thread Joseph Allemandou
Hi All, Everything is back to normal on the cluster, no data loss has been incurred and jobs are up-to-date. You can get back to your normal utilisation ! Thanks Joseph ___ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/m

Re: [Analytics] Issues on Cluster

2016-01-29 Thread Joseph Allemandou
Hi Folks, Data ingestion problem have been solved without data loss, but a lot re-computation is needed. I hope it'll be done over the weekend but in it's not I'll send an email on monday. Have a good weekend ! Joseph On Fri, Jan 29, 2016 at 12:03 PM, Joseph Allemandou < jalleman...@wikimedia.org>

Re: [Analytics] Issues on Cluster

2016-01-29 Thread Joseph Allemandou
Hi, The hadoop cluster is still having issues this morning (ingestion from kafka fails on one partition). Please refrain from using it, it needs all possible resources to catch back :) Thanks Joseph On Thu, Jan 28, 2016 at 2:55 PM, Joseph Allemandou < jalleman...@wikimedia.org> wrote: > Hi again,

Re: [Analytics] Issues on Cluster

2016-01-28 Thread Joseph Allemandou
Hi again, The data ingestion problem is fixed, and the cluster is starting the race against new data coming in. For the moment we are about 20 hours behind on the two big partitions (text and upload), so catching up will take some time. We'll keep you updated as things go along. Cheers Joseph On T

[Analytics] Issues on Cluster

2016-01-28 Thread Joseph Allemandou
Hi Analytics fellows, We are experiencing issues with loading data into the hadoop cluster, therefore blocking the full job pipeline. When fixed, the cluster will be heavily loaded trying to catch up, so please, be nice with it and don't run heavy jobs in the next hours. We'll keep you posted abou