[Cloud] Tools that need user tables joins

2017-12-23 Thread Maarten Dammers
Hi everyone, In the new database setup user databases are no longer possible on the same servers as where the production databases are. I noticed on https://phabricator.wikimedia.org/T142807 Daniel saying "Death blow for GHEL coordinate extraction and WikiMiniAtlas." and on https://phabricato

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Martin Domdey
Hi Maarten! I didn't got it by -announce, but I need it too in close future. Cheers, Martin ... -- Diese Nachricht wurde von meinem Android Mobiltelefon mit GMX Mail gesendet.Am 23.12.2017, 14:29, Maarten Dammers schrieb: Hi everyone, In the new database setup user databases are no longer po

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Maximilian Doerr
So users can no longer create their own DBs now? Or am I missing something? Cyberpower678 English Wikipedia Account Creation Team English Wikipedia Administrator Global User Renamer > On Dec 23, 2017, at 12:44, Martin Domdey wrote: > > Hi Maarten! > > I didn't got it by -announce, but I need

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread John
You can create user databases, however those are hosted on a different server than the replicas, and we cannot join them. On Sat, Dec 23, 2017 at 12:55 PM, Maximilian Doerr < maximilian.do...@gmail.com> wrote: > So users can no longer create their own DBs now? Or am I missing > something? > > Cy

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Maximilian Doerr
Ah. Cyberpower678 English Wikipedia Account Creation Team English Wikipedia Administrator Global User Renamer > On Dec 23, 2017, at 12:58, John wrote: > > You can create user databases, however those are hosted on a different server > than the replicas, and we cannot join them. > > On Sat, De

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Martin Urbanec
You *can* join them, but not by using database logic. You must join them in your application. Martin so 23. 12. 2017 v 19:03 odesílatel Maximilian Doerr < maximilian.do...@gmail.com> napsal: > Ah. > > Cyberpower678 > English Wikipedia Account Creation Team > English Wikipedia Administrator > Glo

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread John
That's not a join. Doing it in the application layer means that both database tables must be fully loaded into RAM to process. Please do not call them the same thing as an application layer "JOIN" and a database join are far from the same thing. On Sat, Dec 23, 2017 at 1:27 PM, Martin Urbanec wro

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Daniel Schwen
Suggesting that joins be just done in the application logic is frankly quite naive. I have one particular application where I need to join millions of entries each on about 50 projects. I used to have an application join algorithm years ago, when the data sets were much smaller and I supported way

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Martin Domdey
why did this happen? Martin -- Diese Nachricht wurde von meinem Android Mobiltelefon mit GMX Mail gesendet.Am 23.12.2017, 19:48, Daniel Schwen schrieb: Suggesting that joins be just done in the application logic is frankly quite naive. I have one particular application where I need to join milli

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread John
why did this happen? Martin See https://phabricator.wikimedia.org/phame/post/view/70/new_wiki_replica_servers_ready_for_use/ ___ Wikimedia Cloud Services mailing list Cloud@lists.wikimedia.org (formerly lab...@lists.wikimedia.org) https://lists.wikimedi

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Daniel Schwen
I do appreciate that the ops team is working to improve reliability and performance of the database access. Unfortunately it seems to me that there is a disconnect between ops and tool devs. I wonder if the ops actually looked at how many user databases have been created and how frequently they got

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Maximilian Doerr
To play devils advocate here they are dedicating and maintaining a service that we get free of charge. Other people have to pay a premium for the service we get. Cyberpower678 English Wikipedia Account Creation Team English Wikipedia Administrator Global User Renamer > On Dec 23, 2017, at 19:2

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Bryan Davis
On Sat, Dec 23, 2017 at 5:28 PM, Daniel Schwen wrote: > I do appreciate that the ops team is working to improve reliability and > performance of the database access. Unfortunately it seems to me that there > is a disconnect between ops and tool devs. I wonder if the ops actually > looked at how ma

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Maximilian Doerr
On a somewhat related note, CentralAuth.labsdb is offline. Any info that can be provided about that? An important bot task is unable to start because of not being able to connect to it. Cyberpower678 English Wikipedia Account Creation Team English Wikipedia Administrator Global User Renamer >

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Bryan Davis
On Sat, Dec 23, 2017 at 9:36 PM, Maximilian Doerr wrote: > On a somewhat related note, CentralAuth.labsdb is offline. Any info that > can be provided about that? An important bot task is unable to start > because of not being able to connect to it. Filed as

Re: [Cloud] interwiki languages

2017-12-23 Thread Martin Domdey
Hi! Does anybody now, what a database table contains a column containing all the interwiki languages existing like 'dewiki', 'enwiki' , 'frwiki', ... Thank you very much, Martin Domdey -- Diese Nachricht wurde von meinem Android Mobiltelefon mit GMX Mail gesendet.Am 24.12.2017, 05:58, Bryan Da

Re: [Cloud] interwiki languages

2017-12-23 Thread Bryan Davis
On Sat, Dec 23, 2017 at 10:03 PM, Martin Domdey wrote: > Hi! > > Does anybody now, what a database table contains a column containing all the > interwiki languages existing like 'dewiki', 'enwiki' , 'frwiki', ... The meta_p `wiki` table may be what you are looking for: $ sql meta Reading table i

Re: [Cloud] interwiki languages

2017-12-23 Thread Martin Domdey
Thank you for your urgent help!   Merry Christmas, Martin ...       Gesendet: Sonntag, 24. Dezember 2017 um 05:44 Uhr Von: "Bryan Davis" An: "Wikimedia Cloud Services general discussion and support" Betreff: Re: [Cloud] interwiki languages On Sat, Dec 23, 2017 at 10:03 PM, Martin Domdey

Re: [Cloud] Tools that need user tables joins

2017-12-23 Thread Gerard Meijssen
Hoi, That is the most awful argument possible. It follows that there is no reason to provide quality functionality in MediaWiki because it is free as well. SHAME on you for going this low. Thanks, GerardM On 24 December 2017 at 03:13, Maximilian Doerr wrote: > To play devils advocate here t