Re: [Cloud] [Cloud-announce] Wiki Replicas 2020 Redesign

Most cross-db JOINs can be recreated using two queries and an external tool to filter the results. However, there are some queries that would be simply impractical due to the large amount of data involved, and the query for overlapping local and Commons images is one of them. There are basically

Re: [Cloud] [Cloud-announce] Wiki Replicas 2020 Redesign

Got it. The https://noc.wikimedia.org/conf/dblists/ lists are plenty fast and easy enough to parse. I'll just cache that. It would be neat if we could rely on the slice specified in meta_p in the future, as in my case we have to query meta_p.wiki regardless, but not a big deal :) Thank you! I

Re: [Cloud] [Cloud-announce] Wiki Replicas 2020 Redesign

On Tue, Nov 10, 2020 at 1:15 PM MusikAnimal wrote: > Hi! Most tools query just a single db at a time, so I don't think this > will be a massive problem. However some such as Global > Contribs[0] and GUC[1] can theoretically query all of them from a single > request. Creating new connections

Re: [Cloud] [Cloud-announce] Wiki Replicas 2020 Redesign

Hi MA, You could still accomplish the local environment you are describing by using 8 ssh tunnels. All the database name DNS aliases go reference the section names, eventually (s1, s2, s3, s4 in the form of s1.analytics.db.svc.eqiad.wmflabs, etc.). An app could be written to connect to the

Re: [Cloud] [Cloud-announce] Wiki Replicas 2020 Redesign

Hi! Most tools query just a single db at a time, so I don't think this will be a massive problem. However some such as Global Contribs[0] and GUC[1] can theoretically query all of them from a single request. Creating new connections on-the-fly seems doable in production, the issue is how to work

[Cloud] [Cloud-announce] Wiki Replicas 2020 Redesign

TLDR: Wiki Replicas' architecture is being redesigned for stability and performance. Cross database JOINs will not be available and a host connection will only allow querying its associated DB. See [1] for more details. Hi!

[Cloud-announce] Wiki Replicas 2020 Redesign

TLDR: Wiki Replicas' architecture is being redesigned for stability and performance. Cross database JOINs will not be available and a host connection will only allow querying its associated DB. See [1] for more details. Hi!

Re: [Cloud] [Cloud-announce] 2020-11-10 ToolsDB (User databases in Toolforge) read-only downtime

This will be happening in around 10 minutes. ToolsDB will be read-only until we can get a consistent dump to rebuild replication. Brooke Storm Staff SRE Wikimedia Cloud Services bst...@wikimedia.org IRC: bstorm > On Nov 6, 2020, at 12:12 PM, Brooke Storm wrote: >

Re: [Cloud-announce] 2020-11-10 ToolsDB (User databases in Toolforge) read-only downtime

This will be happening in around 10 minutes. ToolsDB will be read-only until we can get a consistent dump to rebuild replication. Brooke Storm Staff SRE Wikimedia Cloud Services bst...@wikimedia.org IRC: bstorm > On Nov 6, 2020, at 12:12 PM, Brooke Storm wrote: >