subject:"\[Toolserver\-l\] Future of the toolserver"

On 26/09/12 21:20, Ryan Lane wrote:
> Labs is an infrastructure for building infrastructure. It isn't a
> Toolserver replacement.
> 
> - Ryan

Yet at high level it has been treated as such replacement.
We could also replace the Wikimedia servers with turing machines.


___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

On 26/09/12 20:55, Ryan Lane wrote:
> They can be created in LDAP by making a labsconsole account.
> Additionally, unless the account needs to directly log in via ssh,
> there's no request process needed for the user to be used.
> 
> Of course right now there isn't open registration in labsconsole.
> We're waiting on some Jenkins changes for that to happen. Should be
> any day now.
> 
> Alternatively, the user could be created as a system account via puppet.
> 
> - Ryan

I thought the jenkins changes were ready, after Antoine wrote:
> I have finished the preparation work in Jenkins. It will be deployed as
> soon as we agree on the new workflow, which I still have to write down.


___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-26 Thread Ryan Lane

>> The database servers (replicated and user-databases) are going to be
>> provided by the Labs infrastructure. That's the majority of
>> Toolserver's servers. We have shared storage provided already.
>
> Let me count:
> -2 userland-server (linux)
> -1 userland-server (solaris)
> -2 web-servers (solaris)
> -1 web-serever (linux) (for testing)
> -2 HA-nodes
> -2 aux-servers
> -1 server for the roots
> -1 user-db-server
> -5 replicated db-servers
>
> 6 vs 11 – clearly the majority ;-)
>

This is more of a discussion of capacity. Excluding the databases, the
load of all of those servers would fit on one of our compute nodes as
virtual machines.

Basically what I'm saying is that based on our resources, we should
easily be able to handle the load from the TS community. If it turns
out that we can't, we'll add more capacity.

>>
>> The capacity of Labs is far greater than Toolserver's. We have a
>> cluster in the pmtpa and eqiad datacenters. Each cluster has 7 compute
>> nodes for virtual machines, 2 database servers for replicas, 1
>> database server for user databases and 4 storage nodes for shared
>> storage access.
>
> So you have 2 database servers for 7 wmf-clusters? You are poorer than we are.
>

Well, it's 2 per datacenter for replicas, and 1 per datacenter for
user-databases. That's 6. Also, these are new (and very beefy) servers
with SSDs. Moore's law and all. It could be that we need to add more
servers. If that's the case, we'll do so.

>> It's definitely possible to re-create Toolserver inside of Labs. I'm
>> not suggesting that's what should be done for a migration, but it's an
>> option.
>
> So who is helping me to install solaris? Who is helping me to port our self-
> written software over? Who is helping me to change the self-written scripts?
> Who is helping me to convert our jira-bugs into your bugzilla? Who will update
> all the wiki-pages?
>
> The toolserver is not just a simple Debian with a puppet-system running; its
> much more complex. And while I'm sure that I could re-build something similar
> in labs in a great time-window, it would just be something similar, not the
> same; many tools would break (if you have time, take a look in our JIRA for
> things that are not working on Debian, but on work on Solaris).
>
> Sometimes I have the feeling WMF and WMDE thinks that we can shutdown the TS
> on a Friday, rsync the stuff at the weekend, power-up labs at Monday and
> everything will work….
>

I understand there's a ton of work involved. I'm not suggesting that
it would be a simple task. I'm saying that it's technically possible
to build the same exact environment inside of Labs.

Labs is an infrastructure for building infrastructure. It isn't a
Toolserver replacement.

- Ryan

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-26 Thread Ryan Lane

>> MMPs aren't a very elegant solution. I'd prefer not to
>> force an inelegant solution into a system that allows much more
>> elegant approaches.
>
> Actually, I'm unsure on how to replicate MMP accounts there. It
> shouldn't need to manually create them everywhere. The elegant solution
> is probably to have those accounts in LDAP... Any ideas?
>

They can be created in LDAP by making a labsconsole account.
Additionally, unless the account needs to directly log in via ssh,
there's no request process needed for the user to be used.

Of course right now there isn't open registration in labsconsole.
We're waiting on some Jenkins changes for that to happen. Should be
any day now.

Alternatively, the user could be created as a system account via puppet.

- Ryan

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-26 Thread Daniel Schwen

Another point to consider is that for some projects there is
considerable interdependence (this may not be the norm).
My WikiMiniAtlas for example depends on

1. the OSM database mirror being available
2. the WIWOSM project (although I could proxy that from the TS)
3. Dispensers coordinate extraction database (GHEL)

These dependencies will increase the viscosity of the migration process.
In theory database connections could be ssh tunneled although

1. that may not be a very stable or fast solution
2. someone/somewhat is blocking ssh traffic from labs to the
toolserver cluster (other ssh connections work fine) what is going on
here?!

Dschwen

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

On 26/09/12 12:40, Sumurai8 (DD) wrote:
> 2012/9/26 Platonides:
>> On 26/09/12 08:27, Federico Leva (Nemo) wrote:
 Yes, I know about MMP. It was introduced much later, after people were
 already very used to doing things in an individualized manner. From
 what I'm told MMPs aren't used much.
>>>
>>> So you think that even new users (I believe many users arrived after MMP
>>> were introduced) are not using MMP due to bad old habits of the other
>>> users?
>>> I've had a hard time trying to convince some users to use a MMP; I think
>>> it's not trivial to understand what models actually work and are liked.
>>
>> The reason is as simple as being much more easier, run mkdir and start
>> coding vs open a jira request to get a new MMP for a project which may
>> or may not be interesting (heh, most tools were probably born after
>> coding for a couple of hours after having a good idea) and which nobody
>> is wanting to maintain with you, probably.
>>
> 
> It is not hard to start coding in your own workspace and move code
> from your newly created directory to a MMP. A MMP doesn't necessarely
> have to be maintained by multiple users. While it's convenient having
> someone in the project that can jump in in case your tool breaks badly
> and you are not around to fix it, it is not by any means necessary.

Right. I was explaining why there aren't so many MMPs out there.


___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-26 Thread MZMcBride

Platonides wrote:
> On 25/09/12 20:48, Erik Moeller wrote:
>> 1) WMF is a technology organization. Hosting the core infrastructure
>> for Wikimedia projects is very much what we do. This includes data
>> center operation, monitoring and backups, software deployments,
>> software/service upgrades, code versioning infrastructure, bug
>> tracking infrastructure, additional support systems and services (like
>> this mailing list), etc.
>> 
>> Toolserver is in fact hosted by the Wikimedia Foundation today, in our
>> Amsterdam data-center. We provide space, power and racks for the
>> toolserver cluster, at a cost of about $65,000/year to WMF according
>> to our Director of TechOps.
> 
> Something we should all be grateful for.

I think the current Toolserver setup is less than ideal and I think the
future proposed setup (Tool Labs) is even worse. Right now there's already a
heavy reliance on the goodwill of the Wikimedia Foundation to keep the
Toolserver running. Without database replication, the Toolserver is just
mediocre shared hosting.

Going forward, the situation will worsen, as the Wikimedia Foundation is
basically creating a walled garden. We're watching as the Wikimedia
Foundation puts all of the data, tools, and infrastructure behind the same
organization and then will be able to determine who does and does not have
access to this and under what terms, a step backward as far as I'm
concerned.

Redundancy and duplication in this case is a very good thing, not a bad
thing. If we had ten Toolservers (hosted by Wikimedia chapters, Amazon,
LeaseWeb, or anyone else), it wouldn't be such an issue when database
replication stopped on one of them. It might not be as efficient, but it'd
be a much safer long-term strategy, rather than putting all of the
high-speed access to data (and in some cases, the _only_ access to data
[watchlist, anonymized user preferences, etc.]) behind the Wikimedia
Foundation's control.

MZMcBride

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-26 Thread Andre Koopal

On Wed, Sep 26, 2012 at 10:37:11AM +0200, Pavel Richter wrote:
> Hi Andre,
> 
> 
> 2012/9/26 Andre Koopal :
> > Just replying to the original mail here.
> >
> > While the discussion what the labs project should look like is usefull, the
> > immediate problem at hand is a bit forgotten, that toolserver might end
> > early next year if there is less support.
> 
> Toolserver will not end early next year. Period.
> 
Hi Pavel,

With all due respect, and I do believe your intentions are good, but if you
can't convince DaB about this and he does quit, toolserver will break and
stop. Although Nosy is still there, afaik she is still not up to date about
all the details around toolserver, so there will be a lot of knowledge lost
effectively ending toolserver.

Regards,

Andre

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

Hello Pavel,
At Wednesday 26 September 2012 12:31:52 DaB. wrote:
> With regards to new hardware: Our committment here dos not mean that
> we will simply replace broken parts only; it also means that we will
> buy new stuff, so that the Toolserver can handle increased demand. We
> do have the ressources to do so, and the money is available within our
> annual budget.

so you told me last year and how many servers did we bought this year (quick 
help: less than 1)? All new hardware (with the exception of a simple switch 
and some memory) that was installed this year was bought in 2011. And I told 
WMDE again and again that we are short on hardware. 
Why is it so hard to just add a position to your budget-plan

Toolserver-Hardware xx,000€

? If we will not need all of it in 2013: fine. I will not point at the number 
and demand that all will be spend. The TS will not need much money (2 or 3 new  
database-server would be enough), compared with other projects (like Wikidata, 
Render oder the community-project-budget) or the personal-cost (and you know I 
ALWAYS said that we need personal and it is ok to spend money there!) the 
position would be very small. I just need to be sure that there is money at 
all.

Sincerely,
DaB.

-- 
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885

signature.asc
Description: This is a digitally signed message part.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-26 Thread Sumurai8 (DD)

2012/9/26 Platonides :
> On 26/09/12 08:27, Federico Leva (Nemo) wrote:
>>> Yes, I know about MMP. It was introduced much later, after people were
>>> already very used to doing things in an individualized manner. From
>>> what I'm told MMPs aren't used much.
>>
>> So you think that even new users (I believe many users arrived after MMP
>> were introduced) are not using MMP due to bad old habits of the other
>> users?
>> I've had a hard time trying to convince some users to use a MMP; I think
>> it's not trivial to understand what models actually work and are liked.
>
> The reason is as simple as being much more easier, run mkdir and start
> coding vs open a jira request to get a new MMP for a project which may
> or may not be interesting (heh, most tools were probably born after
> coding for a couple of hours after having a good idea) and which nobody
> is wanting to maintain with you, probably.
>

It is not hard to start coding in your own workspace and move code
from your newly created directory to a MMP. A MMP doesn't necessarely
have to be maintained by multiple users. While it's convenient having
someone in the project that can jump in in case your tool breaks badly
and you are not around to fix it, it is not by any means necessary.

>
>
> Ryan wrote:
>> Toolserver's model is fundamentally different. It's based on an old
>> concept of shared hosting. Labs is built on a model more like a VPS
>> (really more like EC2). Due to that, it's possible to give users far
>> more rights.
>
> labs model didn't exist when the toolserver started. This route was the
> only 'normal' one to follow, specially with a single server.
> It even looks too new for labs right now.
>
>
>
>> If you guys want to build the exact same Toolserver
>> environment as a Labs project, go for it. I have a good feeling you'll
>> start doing things differently when you see the affordances given by
>> having more rights, though.
>
> I have a few ideas on how to improve it on webtools, based on toolserver
> experience, but without big changes.
>
>> MMPs aren't a very elegant solution. I'd prefer not to
>> force an inelegant solution into a system that allows much more
>> elegant approaches.
>
> Actually, I'm unsure on how to replicate MMP accounts there. It
> shouldn't need to manually create them everywhere. The elegant solution
> is probably to have those accounts in LDAP... Any ideas?
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

Hello,
At Wednesday 26 September 2012 12:07:17 DaB. wrote:
> > Even if
> > I would begin this very instance I doubt I would be finish in December
> > 2013; and it would need the same 18 servers (or better: more).
> 
> The database servers (replicated and user-databases) are going to be
> provided by the Labs infrastructure. That's the majority of
> Toolserver's servers. We have shared storage provided already.

Let me count:
-2 userland-server (linux)
-1 userland-server (solaris)
-2 web-servers (solaris)
-1 web-serever (linux) (for testing)
-2 HA-nodes
-2 aux-servers
-1 server for the roots
-1 user-db-server
-5 replicated db-servers

6 vs 11 – clearly the majority ;-)

> 
> The capacity of Labs is far greater than Toolserver's. We have a
> cluster in the pmtpa and eqiad datacenters. Each cluster has 7 compute
> nodes for virtual machines, 2 database servers for replicas, 1
> database server for user databases and 4 storage nodes for shared
> storage access.

So you have 2 database servers for 7 wmf-clusters? You are poorer than we are.

> 
> It's definitely possible to re-create Toolserver inside of Labs. I'm
> not suggesting that's what should be done for a migration, but it's an
> option.

So who is helping me to install solaris? Who is helping me to port our self-
written software over? Who is helping me to change the self-written scripts? 
Who is helping me to convert our jira-bugs into your bugzilla? Who will update 
all the wiki-pages?

The toolserver is not just a simple Debian with a puppet-system running; its 
much more complex. And while I'm sure that I could re-build something similar 
in labs in a great time-window, it would just be something similar, not the 
same; many tools would break (if you have time, take a look in our JIRA for 
things that are not working on Debian, but on work on Solaris).

Sometimes I have the feeling WMF and WMDE thinks that we can shutdown the TS 
on a Friday, rsync the stuff at the weekend, power-up labs at Monday and 
everything will work….

> 
> - Ryan

Sincerely,
DaB.

-- 
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885

signature.asc
Description: This is a digitally signed message part.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

Hello,
At Wednesday 26 September 2012 12:18:12 DaB. wrote:
> (users LOVES joins)

while I'm at joing: We also need a live copy of s4 (commons) on every wmf-db-
cluster (for the image- and commons-people).

Sincerely,
DaB.

-- 
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885


signature.asc
Description: This is a digitally signed message part.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

Hello,
At Wednesday 26 September 2012 11:54:20 DaB. wrote:
> Access to the wmf wiki replicated dbs is public across the entire wmflabs
> network so that's a given within the toollabs project as well.

so the entire labs-cluster use the same database-servers? Do you know that we 
have users that run queries that took hours/days to complete (because there 
are complex, for academic research or just because there is bug in the query)?
And while I read about user-databases in another mail: While the toolserver 
has a dedicated sql-server for that (which REALLY needs 
replacement/extension), many user-databases are on the replicated-db-servers 
(users LOVES joins); some are hundreds of MB big (you can imagine how many 
rows they have). Does labs support that too? 

Sincerely,
DaB.

-- 
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885

signature.asc
Description: This is a digitally signed message part.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-26 Thread Pavel Richter

Hi Andre,


2012/9/26 Andre Koopal :
> Just replying to the original mail here.
>
> While the discussion what the labs project should look like is usefull, the
> immediate problem at hand is a bit forgotten, that toolserver might end
> early next year if there is less support.

Toolserver will not end early next year. Period.

Wikimedia Deutschland will make all necessary investments to keep the
Toolserver up and running, and we will use the next year to discuss
together with the Toolserver users and community a way to transition
into WikiLabs. And I can only urge you all to take active part in the
design and development of Wikilabs, so that it will provide for your
needs.

With regards to new hardware: Our committment here dos not mean that
we will simply replace broken parts only; it also means that we will
buy new stuff, so that the Toolserver can handle increased demand. We
do have the ressources to do so, and the money is available within our
annual budget.
Please support DaB and Sebastian Sooth (who is handling Toolserver
matters from Wikimedia Deutschland side) in putting together a list of
necessary items to be purchased.

What we will not do, however, is ignoring the fact that the Toolserver
will be replaced by Wikilabs sometime in the future. So every
investment we make will have to take this into account.

Pavel

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

On 26/09/12 08:27, Federico Leva (Nemo) wrote:
>> Yes, I know about MMP. It was introduced much later, after people were
>> already very used to doing things in an individualized manner. From
>> what I'm told MMPs aren't used much.
> 
> So you think that even new users (I believe many users arrived after MMP
> were introduced) are not using MMP due to bad old habits of the other
> users?
> I've had a hard time trying to convince some users to use a MMP; I think
> it's not trivial to understand what models actually work and are liked.

The reason is as simple as being much more easier, run mkdir and start
coding vs open a jira request to get a new MMP for a project which may
or may not be interesting (heh, most tools were probably born after
coding for a couple of hours after having a good idea) and which nobody
is wanting to maintain with you, probably.

Ryan wrote:
> Toolserver's model is fundamentally different. It's based on an old
> concept of shared hosting. Labs is built on a model more like a VPS
> (really more like EC2). Due to that, it's possible to give users far
> more rights. 

labs model didn't exist when the toolserver started. This route was the
only 'normal' one to follow, specially with a single server.
It even looks too new for labs right now.

> If you guys want to build the exact same Toolserver
> environment as a Labs project, go for it. I have a good feeling you'll
> start doing things differently when you see the affordances given by
> having more rights, though. 

I have a few ideas on how to improve it on webtools, based on toolserver
experience, but without big changes.

> MMPs aren't a very elegant solution. I'd prefer not to
> force an inelegant solution into a system that allows much more
> elegant approaches.

Actually, I'm unsure on how to replicate MMP accounts there. It
shouldn't need to manually create them everywhere. The elegant solution
is probably to have those accounts in LDAP... Any ideas?

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-26 Thread Andre Koopal

Just replying to the original mail here.

While the discussion what the labs project should look like is usefull, the
immediate problem at hand is a bit forgotten, that toolserver might end
early next year if there is less support.

As others stated, as some of the needed functionality is not present in
labs, having an environment usefull for the current toolserver community
will likely take till end of the year. To then have the community migrate
there, setup the things there, perhaps migrate data, and perhaps change
urls all over the place, will take some time already, so that is likely
middle till end of 2014.

So we will need toolserver up and running for a while, and that will take
investments. And that is the problem that will need immediate attention
at the moment.

Regards,

Andre

On Tue, Sep 25, 2012 at 12:51:50AM +0200, DaB. wrote:
> Hello all,
> 
> in these days WMDE (the chapter that finance the toolserver) is discussing 
> the 
> budget for the next year (2013); you can find it at [1]. At the moment there 
> is 
> no money for new toolserver-hardware in this budget and the CEO Pavel Richter 
> is unwilling to change this ([2] in german) – because he fears that there 
> will 
> be a Wikilabs in 2014. It is not possible for me to run the toolserver for 
> another year with the current hardware – you all know why. For this reason I 
> will request a change of the budget at the general meeting at November, so 
> there will be a vote about. If this vote should fail (and we get no money for 
> new hardware), I am going to retire from my job as root at 30. December 2012.
> I'm not longer able to tolerant the behavior of the german chapter and the 
> WMF 
> in matter of the toolserver; I do this for free and for fun, and it is not 
> longer fun.
> 
> Sincerely,
> DaB.
> 
> P.S: If you are in a board of a chapter that gives money to WMDE for the 
> toolserver: Make sure that it will be spend for hardware. 
> 
> [1] 
> http://meta.wikimedia.org/wiki/Wikimedia_Deutschland/2013_annual_plan_draft/en
> [2] 
> meta.wikimedia.org/wiki/Talk:Wikimedia_Deutschland/2013_annual_plan_draft/de#Toolserver
> 
> -- 
> Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885



> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette


___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-26 Thread Andre Koopal

On Wed, Sep 26, 2012 at 01:59:51AM -0400, Ryan Lane wrote:
> 
> Of course, as I mentioned before, there's nothing forcing this new
> model at all. If you guys want to build the exact same Toolserver
> environment as a Labs project, go for it. I have a good feeling you'll
> start doing things differently when you see the affordances given by
> having more rights, though. Either path chosen, we'll help you with
> infrastructure issues along the way.
> 
> - Ryan
> 
Hi Ryan,

This statement has been made before, but then you put all responsibility
with the community, while the foundation can benefit as well. What people
want and has appeared usefull, is a *supported* environment like toolserver
where people can just get an account, run bots to support a wiki, experiment
with simple tools, if needed work and support a tool together. This can
then be a breeding bed for tools, and if needed brought a stage up, be
documented and pacakged, so it can be rolled out as a wmf supported tool.

That the toolserver environment is supported is imho an essential. While
volunteer admins can certainly help and do a lot of work, including helping
non-unix persons to startup, if there are really problems with the
environment, it better guaranties there is somebody available to acct.

Regards,

Andre Koopal

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver


> Yes, I know about MMP. It was introduced much later, after people were
> already very used to doing things in an individualized manner. From
> what I'm told MMPs aren't used much.

So you think that even new users (I believe many users arrived after MMP 
were introduced) are not using MMP due to bad old habits of the other users?
I've had a hard time trying to convince some users to use a MMP; I think 
it's not trivial to understand what models actually work and are liked.


> Toolserver's model is fundamentally different. It's based on an old
> concept of shared hosting. Labs is built on a model more like a VPS
> (really more like EC2). Due to that, it's possible to give users far
> more rights. MMPs aren't a very elegant solution. I'd prefer not to
> force an inelegant solution into a system that allows much more
> elegant approaches.

I'm not a technical person but yes, I can probably understand that MMP 
is not very elegant. But, do you think users are not using MMP because 
they're not elegant?


Nemo

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

On Wed, Sep 26, 2012 at 1:37 AM, Federico Leva (Nemo)
 wrote:
>> Toolserver's way of doing things is very individualized. It doesn't
>> promote collaboration, and it leads to tools disappearing when a
>> user's account is disabled.
>
> Wrong. Do you know MMP https://wiki.toolserver.org/view/MMP ?
> Toolserver already allows collaboration as you describe it. What you seem to
> be saying is that you want to *force* people to collaborate: are you sure
> this works?
> I'd rather suggest to investigate what works or not in the MMP (very simple)
> model and why people don't use it more in Toolserver, so that you can build
> on that experience and understand better the users' needs.
>

Yes, I know about MMP. It was introduced much later, after people were
already very used to doing things in an individualized manner. From
what I'm told MMPs aren't used much.

Toolserver's model is fundamentally different. It's based on an old
concept of shared hosting. Labs is built on a model more like a VPS
(really more like EC2). Due to that, it's possible to give users far
more rights. MMPs aren't a very elegant solution. I'd prefer not to
force an inelegant solution into a system that allows much more
elegant approaches.

Of course, as I mentioned before, there's nothing forcing this new
model at all. If you guys want to build the exact same Toolserver
environment as a Labs project, go for it. I have a good feeling you'll
start doing things differently when you see the affordances given by
having more rights, though. Either path chosen, we'll help you with
infrastructure issues along the way.

- Ryan

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver


> Toolserver's way of doing things is very individualized. It doesn't
> promote collaboration, and it leads to tools disappearing when a
> user's account is disabled.

Wrong. Do you know MMP https://wiki.toolserver.org/view/MMP ?
Toolserver already allows collaboration as you describe it. What you 
seem to be saying is that you want to *force* people to collaborate: are 
you sure this works?
I'd rather suggest to investigate what works or not in the MMP (very 
simple) model and why people don't use it more in Toolserver, so that 
you can build on that experience and understand better the users' needs.


Nemo

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread Krinkle

On Sep 25, 2012, at 11:12 PM, "DaB."  wrote:
> Hello Erik,
> At Tuesday 25 September 2012 22:24:33 DaB. wrote:
>> 
>> The initial focus for Labs has been to provide functionality that
>> toolserver doesn't - get root on a VM or set of VMs to install/test
>> arbitrary software/services, and get it ready for production
>> deployment.
>> 
> 
> It is nice to have root on a (virtual) machine, but I doubt most tools need it
> [..]
> 

Indeed, however the reason this is crucial for labs is because its scope is much
wider than Toolserver.

For example, in the "deployment" project we simulate nearly the entire WMF
production cluster (including db hosts, apaches, squids, varnish, scalers,
etc.).

This makes one of the very different goals of Labs possible, namely to allow
volunteers to contribute to operations (as opposed to the software we run).

Once everything is puppetized one can basically create a new labs project,
use "wmf-production" as template and instantiate a complete wmf cluster (not
with all the database contents, just the server setup, though it'd contain
sufficient sample data, the purpose is to simulate the servers to develop new
configurations, not use as web site). Give it a subdomain and you'd immediately
have stuff like commons.wikimedia.myproject.wmflabs.org.

Back to the subject, does that mean users will have to learn to manage a VM and
require a public IP and subdomain? No, not at all. We're confusing Dev Labs with
Tool Labs (perhaps we shouldn't name them like that as isolated projects).

Implementation of Tool Labs isn't decided on afaik, but I believe it will 
naturally
solve itself by being distributed among various projects. Behind the scenes they
will likely be a regular labs project, but abstracted for users (e.g. not an
instance-group or even an instance per tool, but all in one instance-group, with
a group of servers for different purposes, like Toolserver has web servers, sql
servers, login/application servers).

E.g. the tools project in wmflabs would have various web servers and application
servers[1]. Users wanting to run queries, bots and long-running/periodic 
processes
would use the application servers. Ideally we'd encourage use of SGE (or
something alike) from the beginning so that the application servers are
optimally used, and it would make it easy to start a process in the background
of an application server from a process on the web server

Access to the wmf wiki replicated dbs is public across the entire wmflabs
network so that's a given within the toollabs project as well.

-- Krinkle

[1] The "bots" project exists already. It doesn't have SGE yet but it's a first
step. There is also a generic "webtools" project being set up as we speak.
Perhaps these two could be merged so that users have shared project storage for
bots generating data to be used by bots and vice-versa.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

On Tue, Sep 25, 2012 at 9:41 PM, Hersfold  wrote:
> I think the vast majority of the people on the Toolserver don't need a lot
> of that extra "beefy" stuff. To run our tools, all we need is:
>
> * A database providing replication from the Foundation's databases.
> * A web server capable of running PHP, CGI, and other web scripts.
> * A linux or solaris machine that we can run bots on.
> * A platform on which all of this runs reliably.
>

Excluding the database replication, all of these things are possible now.

> If any of us needed a Mediawiki installation, we'd have set up our own site
> for it years ago. Sure, I can see you wanting to make options like that
> available to developers, particularly those building extensions to MW, but
> what the Toolserver community - and really, all of the Wikimedia projects -
> need right now is something that meets those four bullet points. With the
> Toolserver down as often as it's up lately, and Labs not ready to provide
> any of that support until December 2013 (not to mention no assistance moving
> there), either a) the Toolserver needs funding so it can get back to working
> order, even if it means no further accounts or projects can be created on
> it, or b) Labs needs to get that stuff running much sooner.
>

Database replication should be done in the next 2-3 months. It may be
done sooner, but that's our current goal date. Brave community members
can start building any other missing infrastructure.

> And frankly, I don't see why WMDE should be footing the bill for all the
> transitioning work when the Foundation isn't providing them any support to
> do so while setting up a competing system at the same time.
>

I've never built Labs to be a competing system. Labs should have
similar functionality, but I find the toolserver portions of Labs to
be a small part of its function. My biggest goal with integrating
Toolserver features is to bring our Toolserver community more closely
together with our developer community. They are fairly disjoint right
now.

WMDE offered to help with the transition work. The decision to move
the community from Toolserver to Labs was made jointly between WMDE
and WMF. It's not possible for WMF to handle building the Labs
infrastructure and to transition the tools. We don't have enough
Toolserver experience to do so properly anyway.

Toolserver was mostly built by the community from the start. Even WMDE
working the transition won't be enough. We need people like you to
help transition. My team will help you as much as needed, and will
gladly answer any questions you have.

- Ryan

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread Chris Grant

We're forgetting something very important here.

Firstly, I would say that we need the toolserver running, for at least 2
more years. My guess is, it will take most of next year, to get labs to a
state where it is a viable replacement that people *want* to switch to;
then the next year as people slowly move over.

However, what we should really be concerned about, is not the hardware, or
how labs is different to the toolserver. It is DaB. If we screw up with
hardware, labs, etc, we can start over, buy new stuff, whatever. DaB we
cannot replace. And if the toolserver needs to keep running for the next
year or two (which it does), then we certainly need DaB. And quite frankly,
that is the real issue at heart here.


-- Chris
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread Hersfold

I think the vast majority of the people on the Toolserver don't need a 
lot of that extra "beefy" stuff. To run our tools, all we need is:


* A database providing replication from the Foundation's databases.
* A web server capable of running PHP, CGI, and other web scripts.
* A linux or solaris machine that we can run bots on.
* A platform on which all of this runs reliably.

If any of us needed a Mediawiki installation, we'd have set up our own 
site for it years ago. Sure, I can see you wanting to make options like 
that available to developers, particularly those building extensions to 
MW, but what the Toolserver community - and really, all of the Wikimedia 
projects - need right now is something that meets those four bullet 
points. With the Toolserver down as often as it's up lately, and Labs 
not ready to provide any of that support until December 2013 (not to 
mention no assistance moving there), either a) the Toolserver needs 
funding so it can get back to working order, even if it means no further 
accounts or projects can be created on it, or b) Labs needs to get that 
stuff running much sooner.


And frankly, I don't see why WMDE should be footing the bill for all the 
transitioning work when the Foundation isn't providing them any support 
to do so while setting up a competing system at the same time.



User:Hersfold
hersfoldw...@gmail.com

On 9/25/2012 9:30 PM, Erik Moeller wrote:

On Tue, Sep 25, 2012 at 2:27 PM, Platonides  wrote:

Hi Platonides.


labs is also a second class citizen.

Moreover, it is explicitely stated not to be for production-like level.

What will happen if a really successful tool reaches to a point where it
de-facto needs it ? (eg. a WLM tool, tools to Request to get an Account
Created, or Request an Unblock appeal...)

Labs doesn't take options away in those cases -- it's just designed to
make it a lot more straightforward to take a tool and integrate it
well, if that's the best path to take for a particular tool.

It opens up additional possibilities:

1) It makes it easy to set up a MediaWiki instance (we're working on a
standard VM config for this to make it really straightforward) so you
can prototype an extension, if that's a direction you want to take for
a particular tool. E.g. some of the above sound like they should be
ideally done as special pages.

I buy into some of the premise stated here (that a lot of tool
developers really don't _want_ to do MediaWiki development), but I
also think it's our job to make it as easy as possible to take
software down the path that makes the most sense.  In contrast, large
web apps like MediaWiki are explicitly prohibited from being made
publicly available per https://wiki.toolserver.org/view/Rules

2) It makes it possible to prototype a standalone service, puppetize
it, and prepare it for production deployment on allocated hardware.

For tools that aren't mission-critical and that are only lightly
integrated, I don't view it as an issue for a tool to run at
tools.wmflabs.org/some/cool/tool (or wherever it's going to live)
indefinitely. (Ryan may want to chip in on that.)


So the tool authors are "on their own", while forced to move out.
What is the WMF *offering*?

We're offering free access to very beefy infrastructure for
Wikimedia-relevant purposes. We're in this for the long haul and
intend to develop Labs into the best environment for testing/staging,
bottom-up experimentation and tool development that we can possibly
build.

We're not able to provide hands-on support for porting stuff over, but
as Ryan suggested, it may be feasible for interested folks to build an
environment in Labs that makes transition easier for toolserver users.
The Labs team (which includes volunteers) is generally very responsive
and helpful, within reason.


Would for instance WMF be willing to lend a few servers to the toolserver until
December 2013?

WMDE has to decide what level of transitional support is appropriate
and budget for it. As for support, we've kicked around the possibility
of them purchasing some additional hardware and us buying it from them
later and using it for other purposes, but we'd have to carefully work
out whether that's feasible in practice.

Erik



___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread Erik Moeller

On Tue, Sep 25, 2012 at 2:27 PM, Platonides  wrote:

Hi Platonides.

> labs is also a second class citizen.
>
> Moreover, it is explicitely stated not to be for production-like level.
>
> What will happen if a really successful tool reaches to a point where it
> de-facto needs it ? (eg. a WLM tool, tools to Request to get an Account
> Created, or Request an Unblock appeal...)

Labs doesn't take options away in those cases -- it's just designed to
make it a lot more straightforward to take a tool and integrate it
well, if that's the best path to take for a particular tool.

It opens up additional possibilities:

1) It makes it easy to set up a MediaWiki instance (we're working on a
standard VM config for this to make it really straightforward) so you
can prototype an extension, if that's a direction you want to take for
a particular tool. E.g. some of the above sound like they should be
ideally done as special pages.

I buy into some of the premise stated here (that a lot of tool
developers really don't _want_ to do MediaWiki development), but I
also think it's our job to make it as easy as possible to take
software down the path that makes the most sense.  In contrast, large
web apps like MediaWiki are explicitly prohibited from being made
publicly available per https://wiki.toolserver.org/view/Rules

2) It makes it possible to prototype a standalone service, puppetize
it, and prepare it for production deployment on allocated hardware.

For tools that aren't mission-critical and that are only lightly
integrated, I don't view it as an issue for a tool to run at
tools.wmflabs.org/some/cool/tool (or wherever it's going to live)
indefinitely. (Ryan may want to chip in on that.)

> So the tool authors are "on their own", while forced to move out.
> What is the WMF *offering*?

We're offering free access to very beefy infrastructure for
Wikimedia-relevant purposes. We're in this for the long haul and
intend to develop Labs into the best environment for testing/staging,
bottom-up experimentation and tool development that we can possibly
build.

We're not able to provide hands-on support for porting stuff over, but
as Ryan suggested, it may be feasible for interested folks to build an
environment in Labs that makes transition easier for toolserver users.
The Labs team (which includes volunteers) is generally very responsive
and helpful, within reason.

> Would for instance WMF be willing to lend a few servers to the toolserver 
> until
> December 2013?

WMDE has to decide what level of transitional support is appropriate
and budget for it. As for support, we've kicked around the possibility
of them purchasing some additional hardware and us buying it from them
later and using it for other purposes, but we'd have to carefully work
out whether that's feasible in practice.

Erik
-- 
Erik Möller
VP of Engineering and Product Development, Wikimedia Foundation

Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

> I doubt that you understand the toolserver. The toolserver-CLUSTER is make of
> 18 different servers. Some run Solaris, some run Debian, some are (big)
> database-servers, some are userland-servers, some are web-servers, some are
> HA-nodes and some are aux-servers (forgetting such details like virtual
> machines, network-devices and external storage). It's a grown infrastructure
> and you can not just move it to labs or rebuild it there as a simple virtual
> machine or cloud-instance. Even if I would begin this very instance I doubt I
> would be finish in December 2013; and it would need the same 18 servers (or
> better: more).
>

The database servers (replicated and user-databases) are going to be
provided by the Labs infrastructure. That's the majority of
Toolserver's servers. We have shared storage provided already.

The capacity of Labs is far greater than Toolserver's. We have a
cluster in the pmtpa and eqiad datacenters. Each cluster has 7 compute
nodes for virtual machines, 2 database servers for replicas, 1
database server for user databases and 4 storage nodes for shared
storage access.

In total that's:

* 14 compute nodes with a total of 2.5TB of RAM, 224 CPU cores, and
14TB of storage for virtual machine images
* 4 database nodes for replicas with a total of 740GB of RAM and 64 CPU cores
* 2 database nodes for user-databases, with a total of 370GB of RAM
and 32 CPU cores
* 8 storage nodes with a total of 128TB of storage

In a project you can create as many instances as you'd like. The
default quota limit per-project is 10 instances, but we can increase
that number as much as needed.

It's definitely possible to re-create Toolserver inside of Labs. I'm
not suggesting that's what should be done for a migration, but it's an
option.

- Ryan

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread DaB.

Hello,
At Wednesday 26 September 2012 00:40:24 DaB. wrote:
> +1. If people really love the Toolserver way of doing things, they
> are more than welcome to re-create the environment as a Labs project.

I doubt that you understand the toolserver. The toolserver-CLUSTER is make of 
18 different servers. Some run Solaris, some run Debian, some are (big) 
database-servers, some are userland-servers, some are web-servers, some are 
HA-nodes and some are aux-servers (forgetting such details like virtual 
machines, network-devices and external storage). It's a grown infrastructure 
and you can not just move it to labs or rebuild it there as a simple virtual 
machine or cloud-instance. Even if I would begin this very instance I doubt I 
would be finish in December 2013; and it would need the same 18 servers (or 
better: more).

Sincerely,
DaB.

P.S: There is no OSM in the 18 servers. OSM is again completely difference 
(Postgres vs. mysql for example).

-- 
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885

signature.asc
Description: This is a digitally signed message part.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

> Both approaches have its merits. The WMF could have copied the
> toolserver recipe much more easily than gooing the route of labs. They
> were brave pursuing the VM based system, and it allows solving different
> problems.
> But both of them have their use. It's not a case where one should "win"
> the other, or the toolserver is "bad" because it is currently hosted by
> WMDE instead of WMF.
>
> Organizations trying to defeat one another will end up with one big
> loser: the community.
>

+1. If people *really* love the Toolserver way of doing things, they
are more than welcome to re-create the environment as a Labs project.

- Ryan

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

On Tue, Sep 25, 2012 at 5:38 PM, Federico Leva (Nemo)
 wrote:
> Indeed the first answers on
> 
> seem to indicate that all tools and services currently on Toolserver will
> just be trashed and people forced to reinvent them from scratch because a
> completely different approach is thought to be better.
>

Toolserver's way of doing things is very individualized. It doesn't
promote collaboration, and it leads to tools disappearing when a
user's account is disabled.

The model in Labs is fundamentally different. Tools, bots, etc. are
meant to be managed by a community, rather than an individual. That
isn't to say that tools and bots can't be created and managed in
exactly the same way as it is in Toolserver, just that it's highly
discouraged.

- Ryan

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread Platonides

On 25/09/12 23:38, Federico Leva (Nemo) wrote:
> Indeed the first answers on
> 
> seem to indicate that all tools and services currently on Toolserver
> will just be trashed and people forced to reinvent them from scratch
> because a completely different approach is thought to be better.
> 
> Nemo

Both approaches have its merits. The WMF could have copied the
toolserver recipe much more easily than gooing the route of labs. They
were brave pursuing the VM based system, and it allows solving different
problems.
But both of them have their use. It's not a case where one should "win"
the other, or the toolserver is "bad" because it is currently hosted by
WMDE instead of WMF.

Organizations trying to defeat one another will end up with one big
loser: the community.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

Indeed the first answers on 
 
seem to indicate that all tools and services currently on Toolserver 
will just be trashed and people forced to reinvent them from scratch 
because a completely different approach is thought to be better.


Nemo

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread Platonides

On 25/09/12 20:48, Erik Moeller wrote:
> 1) WMF is a technology organization. Hosting the core infrastructure
> for Wikimedia projects is very much what we do. This includes data
> center operation, monitoring and backups, software deployments,
> software/service upgrades, code versioning infrastructure, bug
> tracking infrastructure, additional support systems and services (like
> this mailing list), etc.
> 
> Toolserver is in fact hosted by the Wikimedia Foundation today, in our
> Amsterdam data-center. We provide space, power and racks for the
> toolserver cluster, at a cost of about $65,000/year to WMF according
> to our Director of TechOps.

Something we should all be grateful for.

> We also maintain the database replication
> on our end which enables tools to function.

A replication which WMF is already doing for itself, adding an extra
slave hosted by a trusted party doesn't make a techical difference.
While WMF gains a backup server (at some point the toolserver was the
only externally hosted backup). It has also inadvertedly dropped the
binlogs needed by TS when doing server cleanup.

So while very welcome, the WMF side of the db replication is not the
most important piece of the toolserver (having a db copy still makes the
TS better than any other alternative available at the time, though).
The value is probably in how WMDE was able to open that database and
many fruitful tools emerged.

> We can't provide the same level of service for the toolserver
> infrastructure as we do for core operations, and it makes no sense for
> a chapter to build out the required staffing and expertise to do so
> (set up/maintain all or some of the aforementioned functions). Even
> with slightly increased investment, toolserver would always suffer
> from being second or third tier infrastructure.

labs is also a second class citizen.

Moreover, it is explicitely stated not to be for production-like level.

What will happen if a really successful tool reaches to a point where it
de-facto needs it ? (eg. a WLM tool, tools to Request to get an Account
Created, or Request an Unblock appeal...)

> 2) We're not comfortable hosting the toolserver infrastructure as-is.
> There are too many idiosyncratic aspects of its configuration; 

> it has its own wiki,

Since when is that a problem? To take an obvious example, labs also
began by making its own wiki, instead of incorporating itself into an
existing one.

> its own (closed source) version control system, 
The vcs isn't closed source, it's just plain subversion.

The *repository viewer* is closed source, although with a an Open Source
license. I don't know if people really use it too much.

> its own (closed source) issue tracker. 
I'm not a fan of jira, but this is the silliest reason to drop the
toolserver :)

> There are hacks like TUSC that we want
> to replace with better systems/services (e.g. OpenID/OAuth).

OpenID nor OAuth are currently available. If they were, TUSC wouldn't
have been developed in the first time. When they appear, they will
-slowly at first- take over TUSC, as it should.
This ball is at WMF side.

Not to mention, if these «idiosyncratic aspects» were really a problem,
they could easily be changed.

> Chapters are autonomous organizations, and it's WM-DE's call how much
> / whether it wants to continue to invest in infrastructure of any kind
> (and the decision of funding bodies like the FDC to accept or reject
> that proposition). However, for our part, we will not continue to
> support the current arrangement (DB replication, hosting in our
> data-center, etc.) indefinitely.

This sounds as forcing them to go this route.

> The timeline we've discussed with Wikimedia Germany is roughly as follows:
> 
> - Wind down new account creation on toolserver by Q2 of 2013 calendar year
> - Decommission toolserver by December 2013
> 
> WMF can't commit to providing technical support for tool transition
> (there are too many tools), so if there's any area where I think it
> would make sense to ramp up investments on WM-DE's part, it's in
> engineering capacity to support tool developers in porting tools to
> Labs.

So the tool authors are "on their own", while forced to move out.
What is the WMF *offering*?

> That said, there may be a need for emergency purchases/investments to
> keep TS in a usable state until December 2013 (and perhaps allow for
> some buffer room beyond that). That's not our call to make.

But you are refraining WMDE from investing to it now. Would for instance
WMF be willing to lend a few servers to the toolserver until December 2013?

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread DaB.

Hello Erik,
At Tuesday 25 September 2012 22:24:33 DaB. wrote:
> Hi all,
> 
> let me weigh in with a few initial comments, and I'll ask the Labs
> folks to participate here and on Meta as well with regard to technical
> questions.
> 
> The initial focus for Labs has been to provide functionality that
> toolserver doesn't - get root on a VM or set of VMs to install/test
> arbitrary software/services, and get it ready for production
> deployment.

It is nice to have root on a (virtual) machine, but I doubt most tools need 
it. I would also bet that most tool-authors should not have root-access and 
should not be able to install software.
I guess what most people and the WMF and also WMDE does not understand is: 
Most tool-authors are no system-ops or have a Master of Informatic Science. 
Most are amateur programmers who have fun to code a tool and do not maintain 
it once it is done (they also not document it). The tools they are write a 
slow and need way too much resources. They have no backup of their ssh-keys 
and do not extend their accounts in time. And all this is ok. Because (Nosy 
and) I am there to give them a infrastructure they can use, kick them a little 
bit if their tools misbehavior or really need an update, extend their accounts 
and add new ssh-keys. Some times this job sucks, but most times it is a good 
feeling to know that other people use these tools and it helps the wikimedia-
projects.
I very doubt that ANY sysop at WMF would do that (can you imagine Domas 
helping an user to create its first ssh-key (and the second shortly after 
because the user commit the private key instead of the public)?).
I'm very sure that I have users on the TS that are able to use Wikilabs (like 
Merlissimo), but for most users it will be too complex.
> 
> Toolserver is in fact hosted by the Wikimedia Foundation today, in our
> Amsterdam data-center. We provide space, power and racks for the
> toolserver cluster, at a cost of about $65,000/year to WMF according
> to our Director of TechOps. We also maintain the database replication
> on our end which enables tools to function.

It is true that WMF hosts the TS and I am thankful for this. But these costs 
will not go away when the toolserver dies and everyone move to labs – because 
than you have to pay for Labs.
If "we created a mysql-user in the past which took 5 minute and never touched 
it again" is "maintain the database replication on our end " then it's also 
true. The ssh-tunnel to get the data is maintained by our side and most times 
the wmf-sysops are not able to even announce to us when the mysql-master 
changed (we will notice some hours later when our replag increase) or when 
they create a new wiki (we will learn weeks later when an user searches for an 
wiki in our database and can not find it).

> We can't provide the same level of service for the toolserver
> infrastructure as we do for core operations, and it makes no sense for
> a chapter to build out the required staffing and expertise to do so
> (set up/maintain all or some of the aforementioned functions). Even
> with slightly increased investment, toolserver would always suffer
> from being second or third tier infrastructure.

It made sense for years and worked.

> 
> 2) We're not comfortable hosting the toolserver infrastructure as-is.
> There are too many idiosyncratic aspects of its configuration; it has
> its own wiki, its own (closed source) version control system, its own
> (closed source) issue tracker. There are hacks like TUSC that we want
> to replace with better systems/services (e.g. OpenID/OAuth).

Because we use JIRA instead of Bugzilla and Fisheye (which is not a version 
control system BTW) instead of Gerrit the Toolserver has to die? Are you 
kidding? We get these system for free because River asked nicely, but if that 
is really such a problem we can move away from them.
And I was at the tech-meeting in Berlin 3 or 4 years ago where some wmf-techs 
told me, that OpenID would be "next year" – as we can all see the WMF has 
other things to do because there is no OpenID yet. To be clear: We WILL use 
OpenID when it is availablem but we can not use what is not there!

> 
> So, what's next?

What will happen next: I will request a change of the budget for WMDE at the 
general member-meeting of the WMDE.
If my change is accepted there will be some money to extend the toolserver in 
2013 so it can run again properly. Somewhen in 2013 Wikilabs will be moved to 
late 2014 and the TS has to run for another year.
If my change is not accepted, I will retire from the TS at 30. December 2012 
and it is not longer my beer what will happen. Somewhen in January the TS will 
collapse, because of non-maintenance, a wild-running tool or a security 
problem no-one are there to fix. Later this year Wikilabs will be moved to late 
2014. There will be no working plattform for at least two years.

There is also still the possibility that  WMDE comes to senses and changes the 
budget-plan of t

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread Hersfold Wikipedia

+1. Data replication aside, a number of projects depend quite heavily
on the tools hosted on the toolserver. If we won't be able to complete
a transition to labs until 2014, *someone* needs to help keep the
Toolserver supported until then - if WMDE cannot, WMF should supply the
costs needed to maintain what is, at the end of the day, one of their
servers. DaB. also needs support with this, as he's been forced to be
the public face of the degrading server, and subjected to the ire of
all of us as a result (which I have participated in, and apologize for,
DaB. I think we all know you and Nosy are doing the best you can under
the circumstances).

Could the WMF and WMDE please collaborate to work out a plan to keep
the toolserver running as best as it can until labs is fully prepared
to take over?


User:Hersfold
hersfoldw...@gmail.com

Sent from my Windows Phone
From: Merlissimo
Sent: 9/25/2012 16:08
To: Wikimedia Toolserver
Subject: Re: [Toolserver-l] Future of the toolserver
Am 25.09.2012 20:48, schrieb Erik Moeller:
> Toolserver is in fact hosted by the Wikimedia Foundation today, in our
> Amsterdam data-center. [..] We also maintain the database replication
> on our end which enables tools to function.
>
I don't know all internal systems, but i think by maintaining you mean:
grant access to mysql binlogs, traffic costs and sometimes creating a dump.

Why can WMF not administrate the hole database replication servers for
toolserver users in short-term if WMDE should not spend money on this
anymore? Setting up new replication servers at production system is done
quite often. Adding views for hiding private data and adding access
control based on toolserver ldap should be possible. The rest of the
toolserver infrastructure won't be touched by this change.

Currently the replication of database cluster of s3/s6/s7 (all on server
hyacinth) is lagging for more than an hour, performance is very low and
complex queries are taking 10 times longer than normal, so that some of
my queries can not finish within maximum allowed runtime (which brakes
some of my tools since about five days). To solve this problem new
hardware is need. I as toolserver user don't care if support comes from
WMDE or WMF as long as this problem is fixed. I am the one with the
oversized user talk page because other authors asked me why my tools are
not working.

Merlissimo

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread Tim Landscheidt

Merlissimo  wrote:

>> Toolserver is in fact hosted by the Wikimedia Foundation today, in our
>> Amsterdam data-center. [..] We also maintain the database replication
>> on our end which enables tools to function.

> I don't know all internal systems, but i think by
> maintaining you mean: grant access to mysql binlogs, traffic
> costs and sometimes creating a dump.

> Why can WMF not administrate the hole database replication
> servers for toolserver users in short-term if WMDE should
> not spend money on this anymore? Setting up new replication
> servers at production system is done quite often. Adding
> views for hiding private data and adding access control
> based on toolserver ldap should be possible. The rest of the
> toolserver infrastructure won't be touched by this change.
> [...]

Apparently, it is not that smooth for WMF admins
(http://permalink.gmane.org/gmane.org.wikimedia.toolserver/3166):

| > one obvious solution would be to treat the Toolserver database servers
| > as part of WMF's domain so they can use their SOPs (*1) to maintain
| > them instead of manually coordinating with the Toolserver admins (or
| > not).

| I don't think that would be any easier.  Our database systems are
| completely different from theirs, so it would require all WMF admins to
| learn (and remember) a new procedure before they could change anything
| on their side.  Since they usually don't think to notify us of major
| changes in advance, I don't think they would be very happy with that.

Tim

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread Merlissimo


Am 25.09.2012 20:48, schrieb Erik Moeller:

Toolserver is in fact hosted by the Wikimedia Foundation today, in our
Amsterdam data-center. [..] We also maintain the database replication
on our end which enables tools to function.

I don't know all internal systems, but i think by maintaining you mean: 
grant access to mysql binlogs, traffic costs and sometimes creating a dump.


Why can WMF not administrate the hole database replication servers for 
toolserver users in short-term if WMDE should not spend money on this 
anymore? Setting up new replication servers at production system is done 
quite often. Adding views for hiding private data and adding access 
control based on toolserver ldap should be possible. The rest of the 
toolserver infrastructure won't be touched by this change.


Currently the replication of database cluster of s3/s6/s7 (all on server 
hyacinth) is lagging for more than an hour, performance is very low and 
complex queries are taking 10 times longer than normal, so that some of 
my queries can not finish within maximum allowed runtime (which brakes 
some of my tools since about five days). To solve this problem new 
hardware is need. I as toolserver user don't care if support comes from 
WMDE or WMF as long as this problem is fixed. I am the one with the 
oversized user talk page because other authors asked me why my tools are 
not working.


Merlissimo

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

2012-09-25 Thread Erik Moeller

Hi all,

let me weigh in with a few initial comments, and I'll ask the Labs
folks to participate here and on Meta as well with regard to technical
questions.

The initial focus for Labs has been to provide functionality that
toolserver doesn't - get root on a VM or set of VMs to install/test
arbitrary software/services, and get it ready for production
deployment. Labs doesn't have DB replication yet, and it doesn't yet
have an environment optimized for the development of small tools that
are not geared towards deployment in production. There are some
communities within Labs, most notably bots.wmflabs.org, that have
started to optimize their environment for certain categories of tools
(in this case bots).

The second phase of Wikimedia Labs is called "Tool Labs" and its
explicit goal is to be an alternative to the toolserver. This is
outlined here:

https://www.mediawiki.org/wiki/Wikimedia_Engineering/2012-13_Goals#Milestones_by_quarter_2

and here:

https://www.mediawiki.org/wiki/Wikimedia_Labs#Tool_Labs

In other words, it's not suprising that Labs cannot yet function as an
effective toolserver alternative for most purposes, because we've not
started work on the required functionality yet. Our timeline is to do
so beginning in Q1 of the next calendar year. With regard to DB
replication in particular, we're investigating whether we can
accelerate the schedule since it's so highly requested.

It is definitely the goal of ''Tool Labs'' to support the kind of
small-scale, non-deployed tools that toolserver authors love to
create.

Petr Bena, a Labs user, has created this page, which would definitely
benefit from more participation as well:
https://www.mediawiki.org/wiki/Wikimedia_Labs/Toolserver_features_wanted_in_Tool_Labs

It is true that we (WMF) have generally asked chapters to reduce
investment in core infrastructure/services, and specifically asked
WM-DE to work with us in transitioning from toolserver to Labs. There
are a number of reasons for this:

1) WMF is a technology organization. Hosting the core infrastructure
for Wikimedia projects is very much what we do. This includes data
center operation, monitoring and backups, software deployments,
software/service upgrades, code versioning infrastructure, bug
tracking infrastructure, additional support systems and services (like
this mailing list), etc.

Toolserver is in fact hosted by the Wikimedia Foundation today, in our
Amsterdam data-center. We provide space, power and racks for the
toolserver cluster, at a cost of about $65,000/year to WMF according
to our Director of TechOps. We also maintain the database replication
on our end which enables tools to function.

We can't provide the same level of service for the toolserver
infrastructure as we do for core operations, and it makes no sense for
a chapter to build out the required staffing and expertise to do so
(set up/maintain all or some of the aforementioned functions). Even
with slightly increased investment, toolserver would always suffer
from being second or third tier infrastructure.

2) We're not comfortable hosting the toolserver infrastructure as-is.
There are too many idiosyncratic aspects of its configuration; it has
its own wiki, its own (closed source) version control system, its own
(closed source) issue tracker. There are hacks like TUSC that we want
to replace with better systems/services (e.g. OpenID/OAuth).

So, what's next?

Chapters are autonomous organizations, and it's WM-DE's call how much
/ whether it wants to continue to invest in infrastructure of any kind
(and the decision of funding bodies like the FDC to accept or reject
that proposition). However, for our part, we will not continue to
support the current arrangement (DB replication, hosting in our
data-center, etc.) indefinitely.

The timeline we've discussed with Wikimedia Germany is roughly as follows:

- Wind down new account creation on toolserver by Q2 of 2013 calendar year
- Decommission toolserver by December 2013

WMF can't commit to providing technical support for tool transition
(there are too many tools), so if there's any area where I think it
would make sense to ramp up investments on WM-DE's part, it's in
engineering capacity to support tool developers in porting tools to
Labs.

That said, there may be a need for emergency purchases/investments to
keep TS in a usable state until December 2013 (and perhaps allow for
some buffer room beyond that). That's not our call to make.

Hope this clears up some questions around what's going on, and happy
to answer further questions.

All best,
Erik

--
Erik Möller
VP of Engineering and Product Development, Wikimedia Foundation

Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver