Re: [Toolserver-l] Toolserver limitation to come?
Hey, On Wed, 1 May 2013, Ryan Lane wrote: On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt t...@tim-landscheidt.dewrote: | [...] There were never answers to this, so I bring it up here again: 1. How were almost /all/ of the problems the TS have had with replication caused by that redundancy and trying to keep it synced? Thanks for this question :) - I also want to know. From my perspective it does not look like this and even the data inconsistencies appear when we have no commons copy on a mysql instance. And: DaB experimented with federated tables for commons too and we decided to not do this since it does not perform from the start. Probably nowadays when I planned something new in this area (which does not seem to make sense for TS) I'd really give Galera a try - http://codership.com/content/using-galera-cluster 2. What limitation will the Toolserver have at some point? As to #2: From what I've been told this has to do with future sharding plans for the databases, and due to a change in how we'll be doing replication. Of course, I've heard this in passing. For answers to both of these questions you'll need to talk to binasher and/or notpeter on IRC, as they are the ones doing the database work. Thanks for telling... Cheers Marlen/nosy ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Toolserver limitation to come?
On 02/05/13 10:03, Marlen Caemmerer wrote:
Hey,
On Wed, 1 May 2013, Ryan Lane wrote:
On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt
t...@tim-landscheidt.dewrote:
There were never answers to this, so I bring it up here
again:
1. How were almost /all/ of the problems the TS have had
with replication caused by that redundancy and trying
to keep it synced?
Thanks for this question :) - I also want to know.
From my perspective it does not look like this and even the data
inconsistencies appear when we have no commons copy on a mysql instance.
And: DaB experimented with federated tables for commons too and we
decided to not do this since it does not perform from the start.
Probably nowadays when I planned something new in this area (which does
not seem to make sense for TS) I'd really give Galera a try -
http://codership.com/content/using-galera-cluster
FWIW, my {{ref needed}} phrase in the log was also intended to that
statement by Coren, not to multichill reply.
Additionally, I rembember you data inconsistencies happen even with
native mysql replication (as informed by Nosy earlier).
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Toolserver limitation to come?
On May 2, 2013 3:08 AM, Marlen Caemmerer marlen.caemme...@wikimedia.de wrote: Hey, On Wed, 1 May 2013, Ryan Lane wrote: On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt t...@tim-landscheidt.de wrote: | [...] There were never answers to this, so I bring it up here again: 1. How were almost /all/ of the problems the TS have had with replication caused by that redundancy and trying to keep it synced? Ryan, repeaters are from the root of a program inwhich start the initial setup. Thanks for this question :) - I also want to know. From my perspective it does not look like this and even the data inconsistencies appear when we have no commons copy on a mysql instance. And: DaB experimented with federated tables for commons too and we decided to not do this since it does not perform from the start. Probably nowadays when I planned something new in this area (which does not seem to make sense for TS) I'd really give Galera a try - http://codership.com/content/using-galera-cluster 2. What limitation will the Toolserver have at some point? As to #2: From what I've been told this has to do with future sharding plans for the databases, and due to a change in how we'll be doing replication. Of course, I've heard this in passing. For answers to both of these questions you'll need to talk to binasher and/or notpeter on IRC, as they are the ones doing the database work. Thanks for telling... Cheers Marlen/nosy ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Toolserver limitation to come?
Hi,
at the office hour yesterday
(cf. http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20130430.txt):
| [...]
| Coren multichill: The long story short; replicating
| databases is happening soon (Within the month)
| Replicating multiple copies of commons and wikidata
| isn't going to happen that way; it needs to be built
| into application logic or using federated tables.
| Almost /all/ of the problems the TS have had with
| replication were caused by that redundancy and
| trying to keep it synced.
| multichill Coren: So you're basically saying Toollabs is
| useless for me
| Platonides {{ref needed}}
| Coren We're all more than happy to help you (and any other
| maintainer) with adapting your tools to work in that
| setup.
| scfc_de Coren: In that case, Tools will not be able to
| replace Toolserver.
| giftpfla1ze what are federated tables btw?
| Ryan_Lane AFAIK toolserver will also have this limitation
| at some point
| scfc_de Ryan_Lane: What do you mean?
| [...]
There were never answers to this, so I bring it up here
again:
1. How were almost /all/ of the problems the TS have had
with replication caused by that redundancy and trying
to keep it synced?
2. What limitation will the Toolserver have at some point?
Tim
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Toolserver limitation to come?
On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt t...@tim-landscheidt.dewrote:
Hi,
at the office hour yesterday
(cf. http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20130430.txt
):
| [...]
| Coren multichill: The long story short; replicating
| databases is happening soon (Within the month)
| Replicating multiple copies of commons and wikidata
| isn't going to happen that way; it needs to be built
| into application logic or using federated tables.
| Almost /all/ of the problems the TS have had with
| replication were caused by that redundancy and
| trying to keep it synced.
| multichill Coren: So you're basically saying Toollabs is
| useless for me
| Platonides {{ref needed}}
| Coren We're all more than happy to help you (and any other
| maintainer) with adapting your tools to work in that
| setup.
| scfc_de Coren: In that case, Tools will not be able to
| replace Toolserver.
| giftpfla1ze what are federated tables btw?
| Ryan_Lane AFAIK toolserver will also have this limitation
| at some point
| scfc_de Ryan_Lane: What do you mean?
| [...]
There were never answers to this, so I bring it up here
again:
1. How were almost /all/ of the problems the TS have had
with replication caused by that redundancy and trying
to keep it synced?
2. What limitation will the Toolserver have at some point?
As to #2: From what I've been told this has to do with future sharding
plans for the databases, and due to a change in how we'll be doing
replication. Of course, I've heard this in passing. For answers to both of
these questions you'll need to talk to binasher and/or notpeter on IRC, as
they are the ones doing the database work.
- Ryan
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Toolserver limitation to come?
a rel=license href=http://creativecommons.org/licenses/by/3.0/;img
alt=Creative Commons License style=border-width:0 src=
http://i.creativecommons.org/l/by/3.0/88x31.png; //abr /This work is
licensed under a a rel=license href=
http://creativecommons.org/licenses/by/3.0/;Creative Commons Attribution
3.0 Unported License/a.
On May 1, 2013 8:49 PM, Ryan Lane rl...@wikimedia.org wrote:
On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt
t...@tim-landscheidt.dewrote:
Hi,
at the office hour yesterday
(cf.
http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20130430.txt):
| [...]
| Coren multichill: The long story short; replicating
| databases is happening soon (Within the month)
| Replicating multiple copies of commons and wikidata
| isn't going to happen that way; it needs to be built
| into application logic or using federated tables.
| Almost /all/ of the problems the TS have had with
| replication were caused by that redundancy and
| trying to keep it synced.
| multichill Coren: So you're basically saying Toollabs is
| useless for me
| Platonides {{ref needed}}
| Coren We're all more than happy to help you (and any other
| maintainer) with adapting your tools to work in that
| setup.
| scfc_de Coren: In that case, Tools will not be able to
| replace Toolserver.
| giftpfla1ze what are federated tables btw?
| Ryan_Lane AFAIK toolserver will also have this limitation
| at some point
| scfc_de Ryan_Lane: What do you mean?
| [...]
There were never answers to this, so I bring it up here
again:
1. How were almost /all/ of the problems the TS have had
with replication caused by that redundancy and trying
to keep it synced?
2. What limitation will the Toolserver have at some point?
As to #2: From what I've been told this has to do with future sharding
plans for the databases, and due to a change in how we'll be doing
replication. Of course, I've heard this in passing. For answers to both of
these questions you'll need to talk to binasher and/or notpeter on IRC, as
they are the ones doing the database work.
- Ryan
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Toolserver limitation to come?
Remove wiki offline
Do not allow diffrent languages on same feeds
Delete unused accounts
Delete dangerous information
Acknowledge that the sole creator could be anyone, and expects more order
in a hectic database with too much junk being reedited over and over.
Once its written correctly do not allow repeaters to do over incorrectly
work that was ligit.
Figure out how many angles are approaching the database.
This toolserver can cause havoc or can contain order depends on proper
knowledge used in operating it.
Thanks
MILASTAR.TS.RO
On May 1, 2013 9:17 PM, Patricia Pintilie pintilieemp...@gmail.com
wrote:
a rel=license href=http://creativecommons.org/licenses/by/3.0/;img
alt=Creative Commons License style=border-width:0 src=
http://i.creativecommons.org/l/by/3.0/88x31.png; //abr /This work is
licensed under a a rel=license href=
http://creativecommons.org/licenses/by/3.0/;Creative Commons Attribution
3.0 Unported License/a.
On May 1, 2013 8:49 PM, Ryan Lane rl...@wikimedia.org wrote:
On Wed, May 1, 2013 at 5:03 PM, Tim Landscheidt
t...@tim-landscheidt.dewrote:
Hi,
at the office hour yesterday
(cf.
http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20130430.txt):
| [...]
| Coren multichill: The long story short; replicating
| databases is happening soon (Within the month)
| Replicating multiple copies of commons and wikidata
| isn't going to happen that way; it needs to be built
| into application logic or using federated tables.
| Almost /all/ of the problems the TS have had with
| replication were caused by that redundancy and
| trying to keep it synced.
| multichill Coren: So you're basically saying Toollabs is
| useless for me
| Platonides {{ref needed}}
| Coren We're all more than happy to help you (and any other
| maintainer) with adapting your tools to work in that
| setup.
| scfc_de Coren: In that case, Tools will not be able to
| replace Toolserver.
| giftpfla1ze what are federated tables btw?
| Ryan_Lane AFAIK toolserver will also have this limitation
| at some point
| scfc_de Ryan_Lane: What do you mean?
| [...]
There were never answers to this, so I bring it up here
again:
1. How were almost /all/ of the problems the TS have had
with replication caused by that redundancy and trying
to keep it synced?
2. What limitation will the Toolserver have at some point?
As to #2: From what I've been told this has to do with future sharding
plans for the databases, and due to a change in how we'll be doing
replication. Of course, I've heard this in passing. For answers to both of
these questions you'll need to talk to binasher and/or notpeter on IRC, as
they are the ones doing the database work.
- Ryan
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
