alright, that’s very unfortunate – thanks Christian for catching this. All 
these slaves are critical for a variety of scripts that populate dashboards and 
ad-hoc analysis outside of enwiki and dewiki. 

I’ll immediately file an RT ticket.

Dario

On Apr 21, 2014, at 5:27 PM, Toby Negrin <tneg...@wikimedia.org> wrote:

> Hi Christian --
> 
> Thanks for the heads-up. I've verbally notified Dario and the Research and 
> Data team. They will follow up with tech-ops.
> 
> -Toby
> 
> 
> On Mon, Apr 21, 2014 at 3:33 PM, Christian Aistleitner 
> <christ...@quelltextlich.at> wrote:
> Hi,
> 
> one might be tempted to think that the pmtpa data center having its
> servers shut down today should not affect the analytics database
> slaves, as they come with “eqiad” in their name:
> 
>   s[1-7]-analytics-slave.eqiad.wmnet
> 
> .
> 
> But it seems at least our s7 slave went down while pmtpa machines are
> taken down.
> 
> As that looked suspicious, I tried to only look at the first
> indirection of the slaves' hostnames, which unmasked all our slaves
> (except s1, and s5) as being just CNAMEs for pmtpa machines:
> 
> * s2-analytics-slave.eqiad.wmnet (db69.pmtpa.wmnet)
> * s3-analytics-slave.eqiad.wmnet (db71.pmtpa.wmnet)
> * s4-analytics-slave.eqiad.wmnet (db72.pmtpa.wmnet)
> * s6-analytics-slave.eqiad.wmnet (db74.pmtpa.wmnet)
> * s7-analytics-slave.eqiad.wmnet (db68.pmtpa.wmnet)
> 
> The s7 slave is already being unreachable at the time of writing. I
> guess due to the pmpta servers being taken offline. And I expect the
> remaining ones in the above list to go down soon as well.
> 
> I filed RT #7330 about it:
>   https://rt.wikimedia.org/Ticket/Display.html?id=7330
> 
> Not clear when replacements will be available or how to
> proceed. Waiting for a response from Ops.
> 
> Best regards,
> Christian
> 
> 
> --
> ---- quelltextlich e.U. ---- \\ ---- Christian Aistleitner ----
>                            Companies' registry: 360296y in Linz
> Christian Aistleitner
> Gruendbergstrasze 65a        Email:  christ...@quelltextlich.at
> 4040 Linz, Austria           Phone:          +43 732 / 26 95 63
>                              Fax:            +43 732 / 26 95 63
>                              Homepage: http://quelltextlich.at/
> ---------------------------------------------------------------
> 
> _______________________________________________
> Analytics mailing list
> Analytics@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics
> 
> 
> _______________________________________________
> Analytics mailing list
> Analytics@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics

Reply via email to