Re: Debian services and Debian infrastructure
Stephen Gran sg...@debian.org writes: This one time, at band camp, Olivier Berger said: [SNIP] I'm not sure how to encourage new people looking to get involved while simultaneously discouraging this idea that it's a good idea to go away and develop in isolation - that's been a development anti-pattern for a long time, and shouldn't still be something we need to bang on about. In order to try and help, I've added some bits to the wiki, around https://wiki.debian.org/Services [0], hoping that it helps keep DSA in the loop early enough. But that's another problem, about communicating with DSA early enough... and I'm not sure I know where to find the right contact point / procedure ATM. Is it unclear how to contact DSA? Or is there some other part of it that is unclear? I think that https://wiki.debian.org/Teams/DSA is OK, but maybe https://dsa.debian.org/ should be improved ? I'm still not sure if posting on debian-services-admin@l.d.o is a potential way to make sure some educated colleagues (even DSA members) will be able to advice, review, and generally help service prototypes become production-ready. It's a good start. I think the team follows this list. Great then :-) Best regards, [0] https://wiki.debian.org/Services?action=info -- Olivier BERGER http://www-public.telecom-sudparis.eu/~berger_o/ - OpenPGP-Id: 2048R/5819D7E8 Ingenieur Recherche - Dept INF Institut Mines-Telecom, Telecom SudParis, Evry (France) -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87k3d3gpgv@inf-8660.int-evry.fr
Re: Debian services and Debian infrastructure
This one time, at band camp, Olivier Berger said: Stephen Gran sg...@debian.org writes: This one time, at band camp, Olivier Berger said: But that's another problem, about communicating with DSA early enough... and I'm not sure I know where to find the right contact point / procedure ATM. Is it unclear how to contact DSA? Or is there some other part of it that is unclear? I think that https://wiki.debian.org/Teams/DSA is OK, but maybe https://dsa.debian.org/ should be improved ? I'm probably too close - what do you see as lacking? We take patches: http://anonscm.debian.org/gitweb/?p=mirror/dsa-wiki.git Cheers, -- - | ,''`.Stephen Gran | | : :' :sg...@debian.org | | `. `'Debian user, admin, and developer | |`- http://www.debian.org | - signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
Hi, This one time, at band camp, Olivier Berger said: Hi. So IMHO the availability of cloud VMs for people to experiment with (and with root privileges), and on debian.net would be a plus. Now, of course that doesn't mean that the prototypes installed there will be ready for production on DSA managed machines runing stable and with of course every dependencies in real packages. This. I'd like to think about/document repeatable patterns for developers working on applications for deployment on debian systems. I suspect what is going on here is that, until relatively recently, we saw (roughly speaking) the same 20 or 30 people involved in infrastructure development efforts around the project, and there have been an awful lot of unspoken assumptions and things that we've never written down because everyone involved pretty much knew where things are. Looked at from a certain point of view, it's good that we're having these discussions, and even the misunderstandings. It means that new people are getting involved, and we need to help them a bit more. I'd also like to see new people work with us and/or existing teams a bit to learn the existing idioms instead of assuming it's ok to make up new ones, but maybe that's hoping for too much. I'm not sure how to encourage new people looking to get involved while simultaneously discouraging this idea that it's a good idea to go away and develop in isolation - that's been a development anti-pattern for a long time, and shouldn't still be something we need to bang on about. But that's another problem, about communicating with DSA early enough... and I'm not sure I know where to find the right contact point / procedure ATM. Is it unclear how to contact DSA? Or is there some other part of it that is unclear? I'm still not sure if posting on debian-services-admin@l.d.o is a potential way to make sure some educated colleagues (even DSA members) will be able to advice, review, and generally help service prototypes become production-ready. It's a good start. I think the team follows this list. -- - | ,''`.Stephen Gran | | : :' :sg...@debian.org | | `. `'Debian user, admin, and developer | |`- http://www.debian.org | - signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
On 01/04/2014 11:56 PM, Lucas Nussbaum wrote: 3. to provide a place to experiment with new services + create a Debian cloud with virtual machines to develop new services (maybe providing manually-created VMs would be enough -- I'm not sure we need a complex infra such as OpenStack). My first remark about this would be: do we have any other cloud software that can extensively use something like Ceph for distributed storage? Because for me, distributed storage is a mandatory piece which we have to implement when thinking about cloud computing. Otherwise, we have no serious redundancy for storage, and then we have funny issues like what happened with the HDD of Alioth. If the DSA wont go with OpenStack, please name another (comparable) solution that would fit. Also, I hereby propose my help if the DSA need to setup OpenStack. I probably can also involve people from eNovance. They have the expertise to deploy OpenStack using full HA (which I don't know yet). By the way, we need galera in Debian! :) On 01/07/2014 04:47 PM, Wouter Verhelst wrote: Since DSA is using puppet extensively ATM, wouldn't it be better to have a documented procedure on how to set up a VM or chroot or similar environment that uses DSA's puppet recipes to set up a development instance? That way, people can make changes where necessary (while obviously understanding these changes may or may not be acceptable), don't have to worry about making a mistake and killing someone else's machine (after all, it's their own machine), etc. The puppet receipt for setting-up OpenStack are open source, and eNovance actively works on that right now (to have a private cloud product based on just that...). I don't think it's fully production ready right away now, but every day, it's taking a better shape, and I think it's nearly up to speed and could be completely useable in a few weeks. Another thing: I've been providing a bit more than a dozen VMs based on GPLHost Xen solution (I am the main owner of GPLHost, and it's been now more than 10 years...) for various people within the Debian project. The only requirement was that a DD would ask and write his PGP key ID in the registration form, plus the hosted project had to be related to free software in a way or another. Up to now, there's still some space available. I'm not sure if this means that there's not a lot of need for it, or if this is because this is not well known enough. Anyway, I'm very happy I could help, but I'm still a bit frustrated that this isn't really part of the Debian official infrastructure. Cheers, Thomas Goirand (zigo) P.S: Note that I'm not subscribe to -project, which is why I missed this thread, however I'd be very interested in helping. -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52f50b2c.2080...@debian.org
Re: Debian services and Debian infrastructure
On 01/23/2014 06:07 AM, Tollef Fog Heen wrote: Who's then to ensure that machine is secured and kept up to date with security patches and doesn't become a source of spam? Just like for every hosting solution: you deal with it when it happens. With my 10 years experience of running a hosting business, I don't think this is a huge problem. On 01/23/2014 06:07 AM, Tollef Fog Heen wrote: Sorry to say, but most developers are not good sysadmins I'm sure they are better than let's say 90% of my customers. On 01/23/2014 06:07 AM, Tollef Fog Heen wrote: It's sgran who's been thinking about how to do this, but afaik he's seen close to zero interest from developers for it, so it's not happened yet. I don't think we need anything from the DPL as such, but if people are actually interested in something like this happening, saying so would be a good start. I'm convinced as well that there's no demand for it. However, if we start doing some CI things (like piuparts, adequat, rebuild rebuild twice and so on) on each upload, then it makes sense to have a cloud with throwable VMs. And there's all sorts of compute loads that we could make a very good use of. It'd be super nice to have the archive rebuild jobs running on the Debian infrastructure rather than on AWS for example. Cheers, Thomas -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52f50d72.8090...@debian.org
Re: Debian services and Debian infrastructure
On 01/22/2014 12:30 AM, Stephen Gran wrote: I am nervous that your plans to start handing out VMs with no DSA involvement means that in a few years, we'll have a few dozen owned VMs that no one ever bothered to clean up costing us good will with these hosters. What I've done on the VMs which I provided for free to any DD that asked for it, was to have them expire every year. The DDs need to open a support ticket to ask for another year. It worked out well so far. Thomas -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52f50f9c.7040...@debian.org
Re: Debian services and Debian infrastructure
On Sat, 08 Feb 2014, Thomas Goirand wrote: It'd be super nice to have the archive rebuild jobs running on the Debian infrastructure rather than on AWS for example. I agree, and it has been proposed several times over the last few years. To say there was no interest whatsoever would overstate the amount of excitement those suggestions have received. -- | .''`. ** Debian ** Peter Palfrader | : :' : The universal http://www.palfrader.org/ | `. `' Operating System | `-http://www.debian.org/ -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140207171630.gb3...@anguilla.noreply.org
Re: Debian services and Debian infrastructure
- Thomas Goirand z...@debian.org wrote: My first remark about this would be: do we have any other cloud software that can extensively use something like Ceph for distributed storage? Because for me, distributed storage is a mandatory piece which we have to implement when thinking about cloud computing. We use a MooseFS cluster for Brainfood's hosting cluster and use files on the distributed filesystem for our KVM disk images. Moose does not have Ceph's RADOS block device level integration but it does have some very useful features that Ceph does not. For instance, it has the ability to do shallow clones of a file (which Ceph can also do) and does not require that all children be removed before the original parent (a disadvantage which Ceph suffers from). We have found the performance of Moose to be adequate for our needs using an Infiniband interconnect and the reliability has been excellent. If the DSA wont go with OpenStack, please name another (comparable) solution that would fit. We use OpenNebula because we started fairly early and at the time we first began the OpenNebula packaging on Debian was much more mature. That picture may have changed by now. We have considered looking more deeply into OpenStack but OpenNebula is a breeze to script and we do not want to make a decision that is hype based as some accuse the CERN switch to have been. We also have a smaller Eucalyptus installation that we are experimenting with for AWS based services such as Asgard. The puppet receipt for setting-up OpenStack are open source, and eNovance actively works on that right now (to have a private cloud product based on just that...). We have been using Ansible for our host configuration. We first learned of Ansible through our involvement with the Eucalyptus web UI development effort and some family ties between the Eucalyptus team and Ansible. We helped Ansible with some early web development efforts and, as a result, were able to get started with their product earlier in its life. I don't really have anything negative to say about Puppet or Chef but I would say that our use of Ansible has flourished a rate that never happened with either of those products. Another thing: I've been providing a bit more than a dozen VMs based on GPLHost Xen solution (I am the main owner of GPLHost, and it's been now more than 10 years...) for various people within the Debian project. The only requirement was that a DD would ask and write his PGP key ID ... or if this is because this is not well known enough. Anyway, I'm very happy I could help, but I'm still a bit frustrated that this isn't really part of the Debian official infrastructure. Similarly we have handed out some pro-bono virtual machines to various DDs over time just to get their feedback on how our environment worked from their perspective. I would agree that we have not had DDs beating down our door to request this service. If the availability of these kinds of machines was more clearly advertised to DDs that might be a different story. We have also seen some use of the machine for personal services. That is not a problem from our perspective but I can understand how there could be other viewpoints. We would also be happy to collaborate in an effort to organize this kind of service into a more clearly defined Debian cloud platform that provides some well understood set of services (authentication, VPNs, GIT repos, etc.) -- Debian, choice of a GNU generation. -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/8080424.13861391797081490.javamail.r...@newmail.brainfood.com
Re: Debian services and Debian infrastructure
On 07/02/14 at 18:16 +0100, Peter Palfrader wrote: On Sat, 08 Feb 2014, Thomas Goirand wrote: It'd be super nice to have the archive rebuild jobs running on the Debian infrastructure rather than on AWS for example. I agree, and it has been proposed several times over the last few years. To say there was no interest whatsoever would overstate the amount of excitement those suggestions have received. The archive rebuilds are currently running over a very short amount of time ( 8 hours). That's an important feature, because it allows one to rebuild a snapshot of the archive (or a quasi-snapshot, since that could span two dinstalls) and make sure to find all build failures in that snapshot. Unfortunately, to do that, one requires quite a lot of computing power. As a wild guess, I would say 20 to 30 recent servers (I've been using more than 100 EC2 VM for some rebuilds, especially when doing two rebuilds at the same time to compare the rebuilds, e.g. gcc 4.n vs gcc 4.(n+1)). We currently get this computing power for free from Amazon. We used to get it from my employer through a research project (Grid'5000), and we could return to that if needed: the move to Amazon was an attempt at switching from 'only Lucas can work on that' to 'other people can work on that', which was a success given at least David Suarez (general purpose rebuilds + bug filing) and Sylvestre Ledru (clang rebuilds) have been actively running rebuilds over the last year, and I haven't. Also, we made sure of not using anything Amazon-specific in the rebuild infrastructure. Doing those rebuilds on Debian infrastructure would require either: 1) investing in a set of powerful machines to do the rebuilds ($50k as a guess, $2k * 25 -- and that's probably the bare minimum we would need), finding hosting for them, managing them. We get all of that for free from Amazon, so I don't think that this would be good use of Debian's funds. 2) degrading the service: loosen the requirement to rebuild quasi-snapshots of the archive, or use snapshot.debian.org to build a possibly older snapshot of the archive (which would result in filing bugs that were already fixed). None of those options look particularly exciting. That's why I never explored further the idea of running those rebuilds on Debian infrastructure. But maybe you have a nice solution, that you haven't explained yet? One thing we could do, though, is move all the static part of the rebuild infrastructure to Debian infrastructure. That's the virtual machine used to schedule builds on all temporary nodes, and the virtual machine used to store logs. Those logs used to be stored in http://people.d.o/~lucas/, which did not work scale well to several people doing the rebuilds. I approached DSA asking if they would be willing to provide similar archival space in a place accessible by a team including non-DDs. (#debian-admin, 2013-06-03) All of the suggested solutions (mail all build logs as attachments to the bug reports; ask someone else or a cron job to rsync from external host to qa.d.o; push to buildd.d.o) had drawbacks or required additional work that I didn't have time for at the time, so we have just been using an EC2 VM (aws-logs.d.n) to store the logs since then. Probably that should be revisited, but I'm no longer the good point of contact for that (David Suarez is). Lucas -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140208070510.ga27...@xanadu.blop.info
Re: Debian services and Debian infrastructure
On Wed, Jan 22, 2014 at 11:07:49PM +0100, Tollef Fog Heen wrote: Since we discussed that back in June, you added 'providing an OpenStack cloud' on the DSA radar (it noticed that it was mentioned in the 'just starting' category of DSA meeting minutes). It's sgran who's been thinking about how to do this, but afaik he's seen close to zero interest from developers for it, so it's not happened yet. I don't think we need anything from the DPL as such, but if people are actually interested in something like this happening, saying so would be a good start. The lack of feedback worries me a bit, since one one hand it seems like you're pushing quite hard for a cloud and all that, yet we're not seeing a user demand. It could be that people don't approach DSA for whatever reason (if so, we should fix that), but doing lots of work for something we don't know if it's needed doesn't sound like a good use of our time. You're quite right on this. On the other hand, as a random DD, I wouldn't have dared asking DSA to set up a sort of Debian private cloud (which I think would be a good idea in general) without knowing beforehand that DSA had at least some interest in the idea. This is not because I'm scared of approaching DSA or anything such, but because I know that setting up such an infra could be a lot of work. In general, Due to the way improvements are usually achieved in Debian, I'm much more likely to ask fellow DDs to implement small changes (better if I've patches to propose) than to ask them to do a lot of work for my needs. But I certainly understand that DSA is not willing to work on something substantial until you know there is enough demand to make it worth. So, in the end, this sounds like a good match-making situation. If you're worried about demand how about mentioning this in a future DSA mail to d-d-a? (No, I don't think this thread is enough visibility, especially considering the conflictual parts it went through.) Just ask people that would potentially use this service to let you know. I, for one thing, would be interested in something like this. To be clear, I don't have anything which is currently *waiting* for this. Looking back at the services I've recently worked on in Debian: it wouldn't have been a good fit for sources.d.n (due to the large storage requirement), but it would've been for the initial experimental phase of pts.d.n and possibly (depending on how difficult it'll be to set up VMs) also for fire-and-forget VMs we have used in the past as build nodes for BSPs. Just my 0.02 EUR, Cheers. -- Stefano Zacchiroli . . . . . . . z...@upsilon.cc . . . . o . . . o . o Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o Former Debian Project Leader . . @zack on identi.ca . . o o o . . . o . « the first rule of tautology club is the first rule of tautology club » signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
On Wed, 22 Jan 2014, Tollef Fog Heen wrote: 2) the alternative is that they give up on the idea, or host it themselves, which makes it harder to work collaboratively on the service, and results in services that have a single maintainer (or none, in the end). How does having a random VM running in a cloud somewhere make that service easier to integrate into the wider debian.org setup? It means those people who want to experiment get to ask a VM at some predefined places where various parties can monitor who is asking and what their projects are. think it does; what does is the first point: talk to us so we can figure out what works for both sides. Especially given you want to give root to developers on those VMs, there's absolutely nothing that points in a direction of those VMs being more integrateable than a service developed on a developer's own laptop (in a VM or in their regular environment). Indeed, but it gives others (including DSA) an opportunity to chime in sooner. Another requirement is that the developers should have root access on the virtual machine, so that it's easy to try various options without DSA intervention (remember: this is about developement/beta testing, not about running the service in a super-stable, production mode). Who's then to ensure that machine is secured and kept up to date with security patches and doesn't become a source of spam? Sorry to say, but most developers are not good sysadmins and they do not have the tools and infrastructure set up to do systems maintenance well. For this to be an option at all, we'd want to sandbox the machines heavily enough that they will be less attractive machines to develop on than a developer's own laptop. What kind of restrictions do you expect to have to set up? I can imagine some restrictions on outgoing mails (like Amazon VM do by default) but what else could impeded the abilitiy to develop a new service? It's sgran who's been thinking about how to do this, but afaik he's seen close to zero interest from developers for it, so it's not happened yet. I don't think we need anything from the DPL as such, but if people are actually interested in something like this happening, saying so would be a good start. I would be interested by it. I would have used it for pts.debian.net (instead of the Amazon VM we have now). And later, I would like us to be able to use a Debian cloud to automatize the rebuild of reverse dependencies with a set of updated packages. Cheers, -- Raphaël Hertzog ◈ Debian Developer Discover the Debian Administrator's Handbook: → http://debian-handbook.info/get/ -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140123082414.gd3...@x230-buxy.home.ouaza.com
Re: Debian services and Debian infrastructure
]] Stefano Zacchiroli So, in the end, this sounds like a good match-making situation. If you're worried about demand how about mentioning this in a future DSA mail to d-d-a? (No, I don't think this thread is enough visibility, especially considering the conflictual parts it went through.) Just ask people that would potentially use this service to let you know. It was mentioned in our sprint report from last June and has been in the minutes posted later. I don't recall anybody (apart from Lucas) saying at anything at all about it, on the list or in other forums such as IRC. Yes, we could have mentioned it on d-d-a too, but given it's posted both here and mentioned in a Debian news mailing, I think I think it's indicative that there's no great demand based on us not seeing any responses. If people don't express enthusiasm for a service, it's quite likely it might not get worked on. I, for one thing, would be interested in something like this. To be clear, I don't have anything which is currently *waiting* for this. Your interest is noted, thanks! -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87r47zhwrg@qurzaw.varnish-software.com
Re: Debian services and Debian infrastructure
On Wed, Jan 22, 2014 at 11:07:49PM +0100, Tollef Fog Heen wrote: It's sgran who's been thinking about how to do this, but afaik he's seen close to zero interest from developers for it, so it's not happened yet. I don't think we need anything from the DPL as such, but if people are actually interested in something like this happening, saying so would be a good start. o/ I am currently working on http://ci.debian.net (WIP), to run autopkgtest (aka DEP-8) test suites. While developing on my own machine is completely fine, to actually try it out I need external resources even during development because of the demand in processing power: current with only ~200 packages having DEP-8 tests the longest run already took 14h to complete (in a single VM running test suites sequentially). It's currently running the test suites with schroot, but to make sure every test suite runs on a clean environment I will need to switch to using fresh VM's for each test suite. Also parallelizing the runs will be required to have timely results. The proof of concept is running on a Amazon EC2 VM from Debian's credits. I am a little ambivalent on this: on one hand, having this kind of resource available in a DSA-managed environment would have been useful. On the other hand, first prototyping it somewhere where we have free credits helps in knowing how much we will actually need from non-ephemeral, DSA-managed resources when the service gets to a production-ready state, so we might be saving DSA's most precious resource which is people's time. -- Antonio Terceiro terce...@debian.org signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
On 21/01/14 22:09, Tollef Fog Heen wrote: ]] Lucas Nussbaum This mail has been maturing in my in- and outbox for a little while, and it seems like the debate died down until the recent dda mail, so restarting the underlying discussion a bit. Q1. How much should we push for Debian services (services useful to Debian) to be hosted on Debian infrastructure? A lot. Q2. Should we seek other hosting opportunities to ease Debian services development and hosting? Should I authorize the use of Debian money to fund infrastructure not managed by DSA, in the case of a useful service that DSA has been unable to accept in the infrastructure it manages? I don't think so. We've been pretty open with the requirements and the reasoning behind the requirements. If those requirements don't apply for some reason, we should fix the requirements and not work around them. [...] The recurring pattern seems to be that, when DSA is approached to move the service to Debian infrastructure, their evaluation of the service's design results in changes requests or constraints that the service maintainers consider too hard to satisfy. I don't think this is an accurate statement. There have been a few cases where people have shown up with a design we were unable to accomodate and progress have stopped, but in general there's been a tremendous forward progress in what's hosted on DSA infrastructure. I don't think it's ever been easier to get new services hosted on DSA infrastructure in the history of the project. Examples over the last six months are voip, tracker, manpages, contributors, and that's without looking at any of the various static vhosts added. Looking at the SIP implementation a) I've run SIP perfectly well my way (TM) for many years b) when I approached DSA, I discovered there was an alternative to my way c) we discussed, we found common ground, the service is live - the DSA way - which is quite OK, because it is better to have it working in a way that is suitable for a team to support rather than relying on any single individual or vendor As I've mentioned before on this DSA services topic, there are many technical issues that differ from one service to the next, e.g. you can't keep private data, credentials, private keys on some cloud system with a SAN underneath. Copying static web content to mirrors in the cloud is probably quite OK and if some apache log file is lost, who cares - but a Git repository is another thing and it should be on our own hardware. DSA can actually hand some responsibilities out to people to allow them to run things in the cloud but this depends on putting the right interfaces in place and keeping any original data within the official infrastructure. One mechanism we used for RTC is RADIUS (FreeRADIUS server) doing a delegated digest authentication for SIP. With this model, it would be possible to run SIP or TURN servers in the cloud and they would not need a copy of the user database or password hashes. The RADIUS server could potentially be the point where DSA responsibility ends and people can then run other types of service that trust authorized users. If we have to relay a lot of video traffic through TURN, then it would make sense to have local TURN servers in different regions. They keep no local data and can potentially do all access control using RADIUS. Rather than having some awkward discussion about it as a social problem, maybe we can try and look for ways to define such interfaces and give DDs a chance to propose things around them. -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52dfc3af.30...@pocock.com.au
Re: Debian services and Debian infrastructure
Tollef Fog Heen writes (Re: Debian services and Debian infrastructure): This mail has been maturing in my in- and outbox for a little while, and it seems like the debate died down until the recent dda mail, so restarting the underlying discussion a bit. I agree entirely with what Tollef says in his mail. Ian. -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/21215.63955.964954.101...@chiark.greenend.org.uk
Re: Debian services and Debian infrastructure
Hi, disclaimer I'm only replying on the main point I'd like to make, so I'm limited my quoting to that. Do not take it as agreement or disagreement on the other points -- I just consider them more minor points. /disclaimer On 21/01/14 at 22:09 +0100, Tollef Fog Heen wrote: The recurring pattern seems to be that, when DSA is approached to move the service to Debian infrastructure, their evaluation of the service's design results in changes requests or constraints that the service maintainers consider too hard to satisfy. I don't think this is an accurate statement. There have been a few cases where people have shown up with a design we were unable to accomodate and progress have stopped, but in general there's been a tremendous forward progress in what's hosted on DSA infrastructure. I don't think it's ever been easier to get new services hosted on DSA infrastructure in the history of the project. Examples over the last six months are voip, tracker, manpages, contributors, and that's without looking at any of the various static vhosts added. No process is perfect, and while we should look at the cases where we've failed to reach a conclusion leaving everybody happy, I think it's just as, or even more important to look at the cases where we do have a happy ending. 3. to provide a place to experiment with new services + create a Debian cloud with virtual machines to develop new services (maybe providing manually-created VMs would be enough -- I'm not sure we need a complex infra such as OpenStack). In general, we're quite happy to create VMs for people when they show up with a service they want to deploy, so I think this is quite well covered. It's great that you are quite happy to create VMs for people when they show up with a service they want to deploy. But that's not what I'm talking about. I'm seeking a solution for people who show up with an *idea* they want to *experiment* with. Because: 1) that's when they should engage with DSA so that you have a chance to review/participate in the design, not months later when an unofficial service is up and running; 2) the alternative is that they give up on the idea, or host it themselves, which makes it harder to work collaboratively on the service, and results in services that have a single maintainer (or none, in the end). A strong requirement for such a solution is that it should be easy to get access to a VM, even if the developers do not have any working code yet, or are not even set on all the implementation details of the services (e.g.; a 10-lines description of what the service is about should probably be enough). Another requirement is that the developers should have root access on the virtual machine, so that it's easy to try various options without DSA intervention (remember: this is about developement/beta testing, not about running the service in a super-stable, production mode). Since we discussed that back in June, you added 'providing an OpenStack cloud' on the DSA radar (it noticed that it was mentioned in the 'just starting' category of DSA meeting minutes). Providing such a solution on Debian hardware would be a great solution to the above problem. Is there something you need from the DPL to make this a reality? Also, a technical question (I'm curious): is there a technical reason why it's not possible to do that inside the current Ganeti setup? Is there something inherently less secure with providing root access to a VM inside a Ganeti cluster, compared to providing root access to a VM inside an OpenStack cloud? I'm asking because I don't expect a huge demand for this (something between 5 and 15 VM per year), so that's still a scale that could manageable using RT tickets + manual creation. Finally, I'd like to re-state that hosting everything (including development VMs) inside the Debian infrastructure is the most desirable solution. However, it seems that it requires manpower and resources that DSA does not really have for now (I would happy to help if possible with the 'resources' side -- I can't do much about the 'manpower' side). That's something we should work on so that ultimately, e.g. 1 or 2 years from now, we reach that point. But I don't think that we should wait 1 or 2 years to solve that general problem. That's why I'm exploring a compromise and temporary solution, which is using resources from public clouds. I will be very happy to drop that solution as soon as we have a DSA-provided one. Lucas signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
]] Lucas Nussbaum I'm only replying on the main point I'd like to make, so I'm limited my quoting to that. Do not take it as agreement or disagreement on the other points -- I just consider them more minor points. fairy nuff. 3. to provide a place to experiment with new services + create a Debian cloud with virtual machines to develop new services (maybe providing manually-created VMs would be enough -- I'm not sure we need a complex infra such as OpenStack). In general, we're quite happy to create VMs for people when they show up with a service they want to deploy, so I think this is quite well covered. It's great that you are quite happy to create VMs for people when they show up with a service they want to deploy. But that's not what I'm talking about. I'm seeking a solution for people who show up with an *idea* they want to *experiment* with. Because: 1) that's when they should engage with DSA so that you have a chance to review/participate in the design, not months later when an unofficial service is up and running; They don't need a VM to do this, though. 2) the alternative is that they give up on the idea, or host it themselves, which makes it harder to work collaboratively on the service, and results in services that have a single maintainer (or none, in the end). How does having a random VM running in a cloud somewhere make that service easier to integrate into the wider debian.org setup? I don't think it does; what does is the first point: talk to us so we can figure out what works for both sides. Especially given you want to give root to developers on those VMs, there's absolutely nothing that points in a direction of those VMs being more integrateable than a service developed on a developer's own laptop (in a VM or in their regular environment). A strong requirement for such a solution is that it should be easy to get access to a VM, even if the developers do not have any working code yet, or are not even set on all the implementation details of the services (e.g.; a 10-lines description of what the service is about should probably be enough). We expect people who maintain packages to provide their own development environment. How is service development qualitatively different? As I said, we're happy to hand out VMs to people, but as you rightly point out, the time when that happens tend to be when you at least have a proof of concept running, not when you start thinking about how to solve a problem. Another requirement is that the developers should have root access on the virtual machine, so that it's easy to try various options without DSA intervention (remember: this is about developement/beta testing, not about running the service in a super-stable, production mode). Who's then to ensure that machine is secured and kept up to date with security patches and doesn't become a source of spam? Sorry to say, but most developers are not good sysadmins and they do not have the tools and infrastructure set up to do systems maintenance well. For this to be an option at all, we'd want to sandbox the machines heavily enough that they will be less attractive machines to develop on than a developer's own laptop. Since we discussed that back in June, you added 'providing an OpenStack cloud' on the DSA radar (it noticed that it was mentioned in the 'just starting' category of DSA meeting minutes). Providing such a solution on Debian hardware would be a great solution to the above problem. Is there something you need from the DPL to make this a reality? It's sgran who's been thinking about how to do this, but afaik he's seen close to zero interest from developers for it, so it's not happened yet. I don't think we need anything from the DPL as such, but if people are actually interested in something like this happening, saying so would be a good start. The lack of feedback worries me a bit, since one one hand it seems like you're pushing quite hard for a cloud and all that, yet we're not seeing a user demand. It could be that people don't approach DSA for whatever reason (if so, we should fix that), but doing lots of work for something we don't know if it's needed doesn't sound like a good use of our time. Also, a technical question (I'm curious): is there a technical reason why it's not possible to do that inside the current Ganeti setup? Is there something inherently less secure with providing root access to a VM inside a Ganeti cluster, compared to providing root access to a VM inside an OpenStack cloud? I'm asking because I don't expect a huge demand for this (something between 5 and 15 VM per year), so that's still a scale that could manageable using RT tickets + manual creation. Given it's all just KVM, there's little to no difference, security-wise. Finally, I'd like to re-state that hosting everything (including development VMs) inside the Debian infrastructure is the most desirable
Re: Debian services and Debian infrastructure
Hello everybody, I do not think that it is possible to solve all the points of misunderstanding in a long thread of long emails. Personally, I am totally confused by what I read, and do not understand what is even the basic stand point of each participant. May I suggest that you talk directly face to face or by videoconference ? For me, the take home message is: - Do not develop services that need you to have administrator access, because this is hard to transpose on DSA-hosted machines. - Sevices on debian.net should be hosted by Debian as much as possible. After many years as a DD, I became better at using a machine via root privileges than via collective hosting. For instance, I completely forgot how to use CPAN, and I am much more comfortable configuring apache via /etc/apache2/sites-available than via .htaccess files. Also, by my activity of a DD, I am more familiar with Testing-Unstable mixtures than with Stable. For example again, I did the apache 2.4 transition, where the Debian Apache team made a great work, and I would prefer to forget how the 2.2 systems work. I think that if I enjoyed collective hosting, I would have used it through numerus commercial providers, and would not have become a DD. At work, I would happily install software in my home directory on our CentOS servers, and would not mumble regularly that we need access to a cloud system instead. For upstream-metadata.debian.net (still broken, sorry, but I am working on it), I packaged the system (umegaya) so that others can clone it easily if needed, and will be happy to take care myself of the hosting if needed (because I jumped on apache 2.4 too quickly and blends.debian.net is running Wheezy...). But now I have the impression that self-hosting it is very unwelcome, and that the bar for setting up a .debian.net service is very high. I am not asking for an answer now since you need to clarify a lot of points together. I understand the need for transparency, but on the other hand, posting your discussion on -project gives the implicit message that the other subscribers should read it. Please take your time and consider using parallel and more casual discussion channels, to focus the postings on -project to the points of agreement that you reached, rather than the points of disagreement, which we all hope are just transient. Have a nice day, -- Charles -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140122232508.gb12...@falafel.plessy.net
Re: Debian services and Debian infrastructure
]] Charles Plessy Personally, I am totally confused by what I read, and do not understand what is even the basic stand point of each participant. May I suggest that you talk directly face to face or by videoconference ? You're of course free to suggest that, but if you read the initial mail in the thread you'll see that this is an attempt at bringing this discussion into the open, where it belongs. Taking it off to some other medium will work directly against that. For me, the take home message is: - Do not develop services that need you to have administrator access, because this is hard to transpose on DSA-hosted machines. No, because it's a terrible idea in general. If your service requires you to have root on the system, it's probably misdesigned. - Sevices on debian.net should be hosted by Debian as much as possible. Rather, Debian should self-host our own services. What their domain name is is less important. After many years as a DD, I became better at using a machine via root privileges than via collective hosting. For instance, I completely forgot how to use CPAN, and I am much more comfortable configuring apache via /etc/apache2/sites-available than via .htaccess files. Also, by my activity of a DD, I am more familiar with Testing-Unstable mixtures than with Stable. For example again, I did the apache 2.4 transition, where the Debian Apache team made a great work, and I would prefer to forget how the 2.2 systems work. If you look at the various setups, you'll see that it's not uncommon for us to give service admins write access to their vhost setup, so there's hardly any conflict between that and not having root. As for mixing testing and unstable in production, I hope you see why that is not a great idea on a production system. I think that if I enjoyed collective hosting, I would have used it through numerus commercial providers, and would not have become a DD. At work, I would happily install software in my home directory on our CentOS servers, and would not mumble regularly that we need access to a cloud system instead. Installing random software in your ~ instead of from packages is a really bad idea since there's then no central record of what's installed, there's no uniform way to upgrade the packages, etc. All the usual reasons for why we use packages rather than building stuff from source. But now I have the impression that self-hosting it is very unwelcome, and that the bar for setting up a .debian.net service is very high. If you're hosting it yourself, ask two questions: Does this (or will it, in the future) provide a valuable service to Debian? What happens when you lose interest or get run over by a bus? If the answer to the first question is yes (which I hope it is, since else why are you spending your time on it?) you should have a good answer to the second question. My answer is «it runs on Debian infrastructure, so the underlying OS is maintained, the hardware is maintained and we're able to grant other interested DDs access to the service». I think any reasonable answer that satisfies the same conditions basically means you're duplicating the work DSA are already doing, which is inefficient. If you do have a good answer that doesn't imply a waste of resources, where the OS and hardware is maintained and where we, as a project, can have some reasonable level of service contigency expectation even in the case of catastrophic failure of the service owner then I'd like to hear it. I am not asking for an answer now since you need to clarify a lot of points together. I understand the need for transparency, but on the other hand, posting your discussion on -project gives the implicit message that the other subscribers should read it. No, it means we think that other people should be able to read it, which is not the same thing. I have no opinion on how the subscribers of our mailing lists spend their time. Please take your time and consider using parallel and more casual discussion channels, to focus the postings on -project to the points of agreement that you reached, rather than the points of disagreement, which we all hope are just transient. I'm not going to do this for the reasons outlined at the top of this email. If this thread bothers you, just ignore it. Heck, it's not as if -project generally has that much traffic, so just skimming it shouldn't take that many minutes out of your day. I find it really disappointing to hear that you would rather have discussions with project-wide ramifications to be held in secret, rather than out in the open. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/m238kfuq17@rahvafeir.err.no
Re: Debian services and Debian infrastructure
This one time, at band camp, Lucas Nussbaum said: [...] single mail shot with no follow up. Well, it's been a bit over 2 weeks, and you haven't posted a follow up. This doesn't feel like a conversation to me. I understand you're busy, but it feels very much like you're not actually interested in engaging. I find that slightly demoralizing. Cheers, -- - | ,''`.Stephen Gran | | : :' :sg...@debian.org | | `. `'Debian user, admin, and developer | |`- http://www.debian.org | - signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
Hi, On 21/01/14 at 13:11 +, Stephen Gran wrote: This one time, at band camp, Lucas Nussbaum said: [...] single mail shot with no follow up. Well, it's been a bit over 2 weeks, and you haven't posted a follow up. This doesn't feel like a conversation to me. I understand you're busy, but it feels very much like you're not actually interested in engaging. I find that slightly demoralizing. I'm not sure of what you would like me to follow up on? I've just re-checked, and emails from you in that thread did not include a single question. Generally, I agree with everything that has been said in that thread (minor snarky comments from Joerg on m68k and 'real unrealistic requirements', but I didn't think it was worth replying to them). For example, I fully agree that we should try hard to host 'important' services on Debian hardware. Also, I don't think that anybody disagrees that we have a problem with 'speculative' services (as you put it, or 'services development' as I put it). DSA cannot and should not provide hosting for all Debian-related services being developed, as you wrote yourself: I don't think jumping straight to a solution that puts all of the responsibility for every idea for a service in Debian on DSA shoulders is either the only way to go or even a good way to go. There are plenty of bad ideas that should be allowed to wither on the vine, and there are always going to be services that have been designed in such a way as to be difficult to integrate into DSA-managed infrastructure. We are, after all, a reasonably small team of volunteers. Pretending that we can support an infinite number of services or an infinite variety of designs is just going to end in disappointment for someone. Now, of course, I'm very disappointed that nobody from DSA is interested in acting as a gateway between service developers and hosting solutions outside of Debian infrastructure that would be suitable for services in development and experimental/maturing services. In my eyes, that would have been a win-win situation, by putting DSA in the perfect position to be aware of emerging services, and to interact early with service maintainers. But, well, I cannot force anyone to do work that they don't want to do. However, it sounded pointless to argue on that if there is no concrete offer to host Debian's services being developed outside of Debian infrastructure. So, since that discussion, I've been talking to a few hosting providers, and two of them have offered to support Debian with free resources (on their clouds) for Debian development. Since I think that avoiding vendor lock-in is a must, I'd like to make sure that we can get a third one on board before working out further details. That will include deciding how allocation of such resources happen, and where discussion about this should happen. My first choice would be to use debian-services-admin@ for that, but of course that will be your decision as I don't want to 'pollute' the list with traffic you are not interested in. - Lucas signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
This one time, at band camp, Lucas Nussbaum said: Hi, On 21/01/14 at 13:11 +, Stephen Gran wrote: This one time, at band camp, Lucas Nussbaum said: [...] single mail shot with no follow up. Well, it's been a bit over 2 weeks, and you haven't posted a follow up. This doesn't feel like a conversation to me. I understand you're busy, but it feels very much like you're not actually interested in engaging. I find that slightly demoralizing. I'm not sure of what you would like me to follow up on? I've just re-checked, and emails from you in that thread did not include a single question. I didn't think it was a QA session, I thought it was a discussion. You asked several questions, none of which had a completely concrete answer in the thread, although several people offered opinions. Even a mail saying, ok, I take this discussion to mean I should stop talking to DSA and pay for a development environment somewhere would have been useful. I for one didn't take this mail thread to have reached that conclusion. Generally, I agree with everything that has been said in that thread (minor snarky comments from Joerg on m68k and 'real unrealistic requirements', but I didn't think it was worth replying to them). For example, I fully agree that we should try hard to host 'important' services on Debian hardware. Also, I don't think that anybody disagrees that we have a problem with 'speculative' services (as you put it, or 'services development' as I put it). DSA cannot and should not provide hosting for all Debian-related services being developed, as you wrote yourself: I don't think jumping straight to a solution that puts all of the responsibility for every idea for a service in Debian on DSA shoulders is either the only way to go or even a good way to go. There are plenty of bad ideas that should be allowed to wither on the vine, and there are always going to be services that have been designed in such a way as to be difficult to integrate into DSA-managed infrastructure. We are, after all, a reasonably small team of volunteers. Pretending that we can support an infinite number of services or an infinite variety of designs is just going to end in disappointment for someone. I think there's quite a range of options between DSA can't host everything under the sun and I'll go set up a private parallel development environment out of project funds without any further discussion. Now, of course, I'm very disappointed that nobody from DSA is interested in acting as a gateway between service developers and hosting solutions outside of Debian infrastructure that would be suitable for services in development and experimental/maturing services. In my eyes, that would have been a win-win situation, by putting DSA in the perfect position to be aware of emerging services, and to interact early with service maintainers. But, well, I cannot force anyone to do work that they don't want to do. So, I offered at one point to set up an openstack private cloud for DDs to use for service development and so on. I got almost as enthusiastic a response to that as we got to kerberos, AFS, and now MQ. I decided to let it go instead of putting lots of energy into something that no one would use. That sort of thing can be revisited if it's actually interesting for people. I'm not sure what you picture when you talk about us acting as a gateway. Perhaps you could elaborate on that. I'm not keen on playing script monkey to set up machines for people - I'd much rather that interested people be able to do that for themselves. If you just want us to be a point of contact for people developing new services, I think we've said several times that we'd like to be just that. However, it sounded pointless to argue on that if there is no concrete offer to host Debian's services being developed outside of Debian infrastructure. So, since that discussion, I've been talking to a few hosting providers, and two of them have offered to support Debian with free resources (on their clouds) for Debian development. Since I think that avoiding vendor lock-in is a must, I'd like to make sure that we can get a third one on board before working out further details. That will include deciding how allocation of such resources happen, and where discussion about this should happen. My first choice would be to use debian-services-admin@ for that, but of course that will be your decision as I don't want to 'pollute' the list with traffic you are not interested in. No, that's precisely the sort of thing the list is for, I thought - it's not a private list for DSA or anything. Not sure where the word pollute or its scare quotes have come from, but it sure feels hostile. I'll assume you don't mean it that way. I have some operational questions about this cloud setup, since it seems you've delegated running Debian owned machines to us and then gone and got some that you don't want
Re: Debian services and Debian infrastructure
On Tue, Jan 21, 2014 at 03:07:09PM +, Stephen Gran wrote: This one time, at band camp, Lucas Nussbaum said: On 21/01/14 at 13:11 +, Stephen Gran wrote: This one time, at band camp, Lucas Nussbaum said: [...] single mail shot with no follow up. Well, it's been a bit over 2 weeks, and you haven't posted a follow up. This doesn't feel like a conversation to me. I understand you're busy, but it feels very much like you're not actually interested in engaging. I find that slightly demoralizing. I'm not sure of what you would like me to follow up on? I've just re-checked, and emails from you in that thread did not include a single question. I didn't think it was a QA session, I thought it was a discussion. You asked several questions, none of which had a completely concrete answer in the thread, although several people offered opinions. Even a mail saying, ok, I take this discussion to mean I should stop talking to DSA and pay for a development environment somewhere would have been useful. I for one didn't take this mail thread to have reached that conclusion. Indeed. A response would have been appropriate had you reached a decision on how you'd proceed, Lucas. We asked for an open discussion on a public list yet you've proceeded in a non-transparent manner. Generally, I agree with everything that has been said in that thread (minor snarky comments from Joerg on m68k and 'real unrealistic requirements', but I didn't think it was worth replying to them). For example, I fully agree that we should try hard to host 'important' services on Debian hardware. Also, I don't think that anybody disagrees that we have a problem with 'speculative' services (as you put it, or 'services development' as I put it). DSA cannot and should not provide hosting for all Debian-related services being developed, as you wrote yourself: I don't think jumping straight to a solution that puts all of the responsibility for every idea for a service in Debian on DSA shoulders is either the only way to go or even a good way to go. There are plenty of bad ideas that should be allowed to wither on the vine, and there are always going to be services that have been designed in such a way as to be difficult to integrate into DSA-managed infrastructure. We are, after all, a reasonably small team of volunteers. Pretending that we can support an infinite number of services or an infinite variety of designs is just going to end in disappointment for someone. I think there's quite a range of options between DSA can't host everything under the sun and I'll go set up a private parallel development environment out of project funds without any further discussion. Now, of course, I'm very disappointed that nobody from DSA is interested in acting as a gateway between service developers and hosting solutions outside of Debian infrastructure that would be suitable for services in development and experimental/maturing services. In my eyes, that would have been a win-win situation, by putting DSA in the perfect position to be aware of emerging services, and to interact early with service maintainers. But, well, I cannot force anyone to do work that they don't want to do. So, I offered at one point to set up an openstack private cloud for DDs to use for service development and so on. I got almost as enthusiastic a response to that as we got to kerberos, AFS, and now MQ. I decided to let it go instead of putting lots of energy into something that no one would use. That sort of thing can be revisited if it's actually interesting for people. I'm not sure what you picture when you talk about us acting as a gateway. Perhaps you could elaborate on that. I'm not keen on playing script monkey to set up machines for people - I'd much rather that interested people be able to do that for themselves. If you just want us to be a point of contact for people developing new services, I think we've said several times that we'd like to be just that. However, it sounded pointless to argue on that if there is no concrete offer to host Debian's services being developed outside of Debian infrastructure. So, since that discussion, I've been talking to a few hosting providers, and two of them have offered to support Debian with free resources (on their clouds) for Debian development. Since I think that avoiding vendor lock-in is a must, I'd like to make sure that we can get a third one on board before working out further details. That will include deciding how allocation of such resources happen, and where discussion about this should happen. My first choice would be to use debian-services-admin@ for that, but of course that will be your decision as I don't want to 'pollute' the list with traffic you are not interested in. No, that's precisely the sort of thing the list is for, I thought - it's not a
Re: Debian services and Debian infrastructure
On 21/01/14 at 15:07 +, Stephen Gran wrote: I think there's quite a range of options between DSA can't host everything under the sun and I'll go set up a private parallel development environment out of project funds without any further discussion. I don't know who you are quoting here, but I never said that I would go set up a private parallel development environment out of project funds. What I'm doing currently is what I wrote in the mail you replied to: However, it sounded pointless to argue on that if there is no concrete offer to host Debian's services being developed outside of Debian infrastructure. So, since that discussion, I've been talking to a few hosting providers, and two of them have offered to support Debian with free resources (on their clouds) for Debian development. Since I think that avoiding vendor lock-in is a must, I'd like to make sure that we can get a third one on board before working out further details. That is, *explore the possiblity* to get *free* resources for Debian development. Now, of course, I'm very disappointed that nobody from DSA is interested in acting as a gateway between service developers and hosting solutions outside of Debian infrastructure that would be suitable for services in development and experimental/maturing services. In my eyes, that would have been a win-win situation, by putting DSA in the perfect position to be aware of emerging services, and to interact early with service maintainers. But, well, I cannot force anyone to do work that they don't want to do. So, I offered at one point to set up an openstack private cloud for DDs to use for service development and so on. I got almost as enthusiastic a response to that as we got to kerberos, AFS, and now MQ. I decided to let it go instead of putting lots of energy into something that no one would use. That sort of thing can be revisited if it's actually interesting for people. FTR, I agree that the demand will not be huge for that. Probably in the order of 4-8 services hosting requests per year. However, I disagree that no one would use that. I know of at least two services using VMs on EC2 for service development currently. If we can avoid spending DSA's time on a private OpenStack Cloud by getting free resources, and solve the problem of providing a development infrastructure that way, I really think that this is the way to go. I'm not sure what you picture when you talk about us acting as a gateway. Perhaps you could elaborate on that. I'm not keen on playing script monkey to set up machines for people - I'd much rather that interested people be able to do that for themselves. If you just want us to be a point of contact for people developing new services, I think we've said several times that we'd like to be just that. I think that the tasks involved would be something such as: - Work with hosting providers so that we have some diversity in the offerings, and avoid vendor lock-in. - Understand the offerings (what's available? possible?) - Document the various offerings (wiki page) - Understand what prospective service maintainers are asking for, in terms of resources. Maybe provide advice on design. - Do the matching between what service maintainers need and the hosting providers - Maybe provide some support to service maintainers (e.g. for dealing with the specifics of each hosting provider) - Maybe make it possible to install DSA-flavored VMs on the various Clouds, using the resources pointed to in [1]. [1] http://lists.debian.org/20140109083128.ga12...@varinia.lobefin.net There's nothing here that *needs* to be done by DSA, but it would certainly be very helpful to have DSA members involved. However, it sounded pointless to argue on that if there is no concrete offer to host Debian's services being developed outside of Debian infrastructure. So, since that discussion, I've been talking to a few hosting providers, and two of them have offered to support Debian with free resources (on their clouds) for Debian development. Since I think that avoiding vendor lock-in is a must, I'd like to make sure that we can get a third one on board before working out further details. That will include deciding how allocation of such resources happen, and where discussion about this should happen. My first choice would be to use debian-services-admin@ for that, but of course that will be your decision as I don't want to 'pollute' the list with traffic you are not interested in. No, that's precisely the sort of thing the list is for, I thought - it's not a private list for DSA or anything. Not sure where the word pollute or its scare quotes have come from, but it sure feels hostile. I'll assume you don't mean it that way. Well, if DSA has nothing to do with this development infrastructure they don't host, it could have made sense to have a separate list for that, just to keep the signal/noise ratio high
Re: Debian services and Debian infrastructure
On 21/01/14 at 15:47 +, Luca Filipozzi wrote: And I actively engaged other DDs regarding their VPS opportunities (although no response so far in some cases). Oh, that's great! I've just sent you a private mail to share my contacts. I really had no idea that you intended to work on that, given that when we discussed related topics in June and again in December, you showed little interest in exploring such setups. It would have avoided some duplicate work if you mentioned it when I talked about it in [1]: | As you know, Amazon provides Debian with free credits on the | AWS Cloud. Those credits are normally used for archive rebuilds, but | have already been used by a GSOC project (PTS rewrite), and more | recently, to start work on an archive-wide autopkgtest setup. A few more | virtual machines could be accomodated, and I will investigate if we | could extend that even more. Also, codesearch.d.n is hosted on | Rackspace's Cloud[1]. I will also follow up on that. [1] https://lists.debian.org/debian-project/2014/01/msg00010.html Lucas signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
This one time, at band camp, Lucas Nussbaum said: On 21/01/14 at 15:07 +, Stephen Gran wrote: I think there's quite a range of options between DSA can't host everything under the sun and I'll go set up a private parallel development environment out of project funds without any further discussion. I don't know who you are quoting here, but I never said that I would go set up a private parallel development environment out of project funds. I think I might not have said that clearly enough or something, since I did not accuse you of doing that. We've very nearly finished cleaning up an awful lot of misunderstanding and lost good will from various donors because many Joe DDs asked for donations in the name of the project and then used it for a personal development platform without coordinating anything with us. I'd rather not start that up again. I am nervous that your plans to start handing out VMs with no DSA involvement means that in a few years, we'll have a few dozen owned VMs that no one ever bothered to clean up costing us good will with these hosters. I assume you have plans to prevent that, but as you didn't talk to us I can't know them yet. I also know from experience that we will be the people contacted in the event of trouble, and if we have no idea what is going on in these environments, it wastes time and effort. What I'm doing currently is what I wrote in the mail you replied to: However, it sounded pointless to argue on that if there is no concrete offer to host Debian's services being developed outside of Debian infrastructure. So, since that discussion, I've been talking to a few hosting providers, and two of them have offered to support Debian with free resources (on their clouds) for Debian development. Since I think that avoiding vendor lock-in is a must, I'd like to make sure that we can get a third one on board before working out further details. That is, *explore the possiblity* to get *free* resources for Debian development. Great. I'm happy you've been successful soliciting donations. I'm less happy about the decision to run with it instead of talking about it. If we can avoid spending DSA's time on a private OpenStack Cloud by getting free resources, and solve the problem of providing a development infrastructure that way, I really think that this is the way to go. That is something that might have been interesting to discuss, if only we had had a discussion. I have some operational questions about this cloud setup, since it seems you've delegated running Debian owned machines to us and then gone and got some that you don't want us to run. I'm not sure what to do with that disjuncture. I don't know what you are talking about. Where did I got some Debian owned machines that I don't want DSA to run? Are these resources being given to Debian or to lucas? I thought you meant you were getting this stuff for Debian? If they're given to Debian, are they not Debian machines? Not that I'm saying we have to run them, but I'm saying that you seem to have made the decision about how this is going without talking to the team that you delegated to do exactly this job. Cheers, -- - | ,''`.Stephen Gran | | : :' :sg...@debian.org | | `. `'Debian user, admin, and developer | |`- http://www.debian.org | - signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
]] Lucas Nussbaum On 21/01/14 at 15:07 +, Stephen Gran wrote: I think there's quite a range of options between DSA can't host everything under the sun and I'll go set up a private parallel development environment out of project funds without any further discussion. I don't know who you are quoting here, but I never said that I would go set up a private parallel development environment out of project funds. If you think that's what sgran wrote, you might want to read again. [...] I have some operational questions about this cloud setup, since it seems you've delegated running Debian owned machines to us and then gone and got some that you don't want us to run. I'm not sure what to do with that disjuncture. I don't know what you are talking about. Where did I got some Debian owned machines that I don't want DSA to run? Those VMs are machines. They're not owned by the individual developers (quite obviously), so they're owned by Debian. The DSA delegation you sent less than two weeks ago include: - Setting up and administering Debian-owned machines, ensuring that they are kept secure, operational, and running. so it's quite clear to me at least than any VMs owned by Debian falls under that umbrella. Separately, given how upset you got after having a request to add a member to the DSA team Cc-ed to debian-project, I think you're completely out of bounds when you're informing DSA (and others) that you're working on setting up parallel infrastructure by mailing debian-devel-announce. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87k3dte21y@xoog.err.no
Re: Debian services and Debian infrastructure
On Tue, Jan 21, 2014 at 05:22:28PM +0100, Lucas Nussbaum wrote: On 21/01/14 at 15:47 +, Luca Filipozzi wrote: And I actively engaged other DDs regarding their VPS opportunities (although no response so far in some cases). Oh, that's great! I've just sent you a private mail to share my contacts. I really had no idea that you intended to work on that, given that when we discussed related topics in June and again in December, you showed little interest in exploring such setups. It would have avoided some duplicate work if you mentioned it when I talked about it in [1]: | As you know, Amazon provides Debian with free credits on the | AWS Cloud. Those credits are normally used for archive rebuilds, but | have already been used by a GSOC project (PTS rewrite), and more | recently, to start work on an archive-wide autopkgtest setup. A few more | virtual machines could be accomodated, and I will investigate if we | could extend that even more. Also, codesearch.d.n is hosted on | Rackspace's Cloud[1]. I will also follow up on that. [1] https://lists.debian.org/debian-project/2014/01/msg00010.html My conversations were more in the context of finding resources for snapshot.debian.org storage (which means one VPS with lots of storage) than in the context of finding generic virtual machines (cutely, my tweet garnered more response than my other actions). The more generic one you were carbon copied on. I suggest that your private discussions with VM providers should have been discussed with (or at least disclosed to) DSA. (There is little value in this you should have tit for tat so I'll stop here.) I'm worried that we're heading down a path that we've seen before: individual DDs obtain resources under the banner of Debian with single DD access and with no record of the donation. As DPL, you are acting officially compared to the above but this is what I meant by fractured landscape. Not all my colleagues want to be involved, to be sure, in managing VPS resources but having them documented and allocated (ie managed) is important. I don't think that that is a (useful) DPL task. So, if it isn't DSAs, make it Auditors or somebody's. In other words: don't fracture the landscape. But if you're going to, make sure it's managed. -- Luca Filipozzi http://www.crowdrise.com/SupportDebian signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
On 21/01/14 at 17:23 +0100, Tollef Fog Heen wrote: I have some operational questions about this cloud setup, since it seems you've delegated running Debian owned machines to us and then gone and got some that you don't want us to run. I'm not sure what to do with that disjuncture. I don't know what you are talking about. Where did I got some Debian owned machines that I don't want DSA to run? Those VMs are machines. They're not owned by the individual developers (quite obviously), so they're owned by Debian. The DSA delegation you sent less than two weeks ago include: - Setting up and administering Debian-owned machines, ensuring that they are kept secure, operational, and running. so it's quite clear to me at least than any VMs owned by Debian falls under that umbrella. Separately, given how upset you got after having a request to add a member to the DSA team Cc-ed to debian-project I think you're completely out of bounds when you're informing DSA (and others) that you're working on setting up parallel infrastructure by mailing debian-devel-announce. Given that, in [3], I wrote: | As you know, Amazon provides Debian with free credits on the | AWS Cloud. Those credits are normally used for archive rebuilds, but | have already been used by a GSOC project (PTS rewrite), and more | recently, to start work on an archive-wide autopkgtest setup. A few more | virtual machines could be accomodated, and I will investigate if we | could extend that even more. Also, codesearch.d.n is hosted on | Rackspace's Cloud[1]. I will also follow up on that. I think that I informed the project that I would explore opportunities to host services in development using free resources provided by public clouds. So, we can bikeshed for a long time over whether free credits on public Clouds are machines that must be managed by DSA. And we should probably throw in the discussion the AWS credits donated to Debian (see [1]) -- however I'm surprised that this comes up now when we have had such credits since 2011[2]. Or, we could try to get constructive, and get things done. I had never planned to keep the management of this in DPL-dom, and even asked DSA in [3] if you would like to manage those resources. My second-choice would have been to build a small team to do the tasks listed in [4]. But if DSA would like to do those tasks, I'm super-happy to hand them over to you. My question from [3] on whether you would like to be in charge of that still stands, and I was planning to get back to you about it once I had clarified if it was possible to get such a scheme set up with Cloud providers. So, just let me know. [1] https://lists.debian.org/debian-devel-announce/2013/12/msg3.html [2] https://lists.debian.org/debian-qa/2011/11/msg1.html [3] https://lists.debian.org/debian-project/2014/01/msg00010.html [4] https://lists.debian.org/debian-project/2014/01/msg00089.html Lucas -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20140121171426.ga26...@xanadu.blop.info
Re: Debian services and Debian infrastructure
]] Lucas Nussbaum This mail has been maturing in my in- and outbox for a little while, and it seems like the debate died down until the recent dda mail, so restarting the underlying discussion a bit. Q1. How much should we push for Debian services (services useful to Debian) to be hosted on Debian infrastructure? A lot. Q2. Should we seek other hosting opportunities to ease Debian services development and hosting? Should I authorize the use of Debian money to fund infrastructure not managed by DSA, in the case of a useful service that DSA has been unable to accept in the infrastructure it manages? I don't think so. We've been pretty open with the requirements and the reasoning behind the requirements. If those requirements don't apply for some reason, we should fix the requirements and not work around them. [...] The recurring pattern seems to be that, when DSA is approached to move the service to Debian infrastructure, their evaluation of the service's design results in changes requests or constraints that the service maintainers consider too hard to satisfy. I don't think this is an accurate statement. There have been a few cases where people have shown up with a design we were unable to accomodate and progress have stopped, but in general there's been a tremendous forward progress in what's hosted on DSA infrastructure. I don't think it's ever been easier to get new services hosted on DSA infrastructure in the history of the project. Examples over the last six months are voip, tracker, manpages, contributors, and that's without looking at any of the various static vhosts added. No process is perfect, and while we should look at the cases where we've failed to reach a conclusion leaving everybody happy, I think it's just as, or even more important to look at the cases where we do have a happy ending. 1. to improve communication between DSA and service maintainers + have an archived, public list where (prospective) service maintainers can engage with DSA about stuff they are planning/thinking about. (Maybe recycle debian-admin@l.d.o, or use debian-services-admin@l.d.o?) We've already talked about making d-a@lists public (but not older archives). It's basically just somebody doing the work. [...] 2. to improve the tracking of services + request wiki pages from maintainers that detail who are the contact points, what the service does, what are its requirements. State a service level agreement (informative of expectations, not punitive, of course) + have service maintainers create similar pages for services in development, to provide some kind of incubation process during which DSA can provide feedback. Wiki pages are terrible for discussion, but having wiki pages (or similar) for all our services, contact points and so on might be useful. In practice, I think we're doing ok for any official services since we informally know who runs a given service, though. 3. to provide a place to experiment with new services + create a Debian cloud with virtual machines to develop new services (maybe providing manually-created VMs would be enough -- I'm not sure we need a complex infra such as OpenStack). In general, we're quite happy to create VMs for people when they show up with a service they want to deploy, so I think this is quite well covered. [...] Q1. How much should we push for Debian services (services useful to Debian) to be hosted on Debian infrastructure? Fragmentation in terms of hosting is harmful, and we should all try to get our services hosted on Debian infrastructure, because that's the easiest way to ensure that the maintainance of such services can be shared or transferred between developers. Now, for some services, it seems that doing this has become so hard that it harmed the service itself, and badly demotivated the service maintainers. I don't think that we should allow that to happen. We should stay open to other hosting solutions, when this is necessary. I don't think we should, simply because it either means one of two things: a) DSA are too difficult to work with and our standards are too high/too difficult to satisfy, in which case we should fix reality and our standards to match up. b) the service is in bad enough shape that it shouldn't be deployed on any infrastructure, since keeping it running will cost too much (in terms of manpower, cpu power or whatever). Deploying it on non-d.o infrastructure, as what you're describing it as, it to route around DSA and we as a project have a problem when the maintainer loses interest (and then often this becomes DSA's problem, but with the additional problem of there being no design or operation documentation, hard to get in touch with whoever runs the infrastructure, security problems due to lack of maintenance, etc) Question to DSA: would you be willing to manage the allocation of virtual machines on such
Re: Debian services and Debian infrastructure
This one time, at band camp, Wouter Verhelst said: Op 05-01-14 14:28, Joerg Jaspert schreef: On 13446 March 1977, Lucas Nussbaum wrote: I'm worried that this situation is harmful for the project. Yes, seperate development with different requirements than the production environment is harmful. It's a lesson that every larger company with an IT Department goes through at some point - and it's the same for our project. You need a development environment that is build and handled the same way as the production environment. Sure you need to evolve that, it may not stand still, with new requirements coming along, but you need to do that in close work with the people maintaining the production one, to ensure that the way thinks evolve is actually something that can be mirrored in the area it is intended to be run later. (Alternatively you can have a free-for-all run development and then get a third env, in which you then try to get to run the new release in a (newer version of the) production env). Since DSA is using puppet extensively ATM, wouldn't it be better to have a documented procedure on how to set up a VM or chroot or similar environment that uses DSA's puppet recipes to set up a development instance? That way, people can make changes where necessary (while obviously understanding these changes may or may not be acceptable), don't have to worry about making a mistake and killing someone else's machine (after all, it's their own machine), etc. If you prefer to run your own VM or chroot but want it to look like a DSA machine as much as possible, we do have public documentation on how we set things up. Our puppet is public: http://git.debian.org/?p=mirror/dsa-puppet.git As are our new machine setup notes: https://dsa.debian.org/howto/new-machine/ https://dsa.debian.org/howto/puppet-setup/ We're usually happy to merge patches. Cheers, -- - | ,''`.Stephen Gran | | : :' :sg...@debian.org | | `. `'Debian user, admin, and developer | |`- http://www.debian.org | - signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
Op 05-01-14 14:28, Joerg Jaspert schreef: On 13446 March 1977, Lucas Nussbaum wrote: I'm worried that this situation is harmful for the project. Yes, seperate development with different requirements than the production environment is harmful. It's a lesson that every larger company with an IT Department goes through at some point - and it's the same for our project. You need a development environment that is build and handled the same way as the production environment. Sure you need to evolve that, it may not stand still, with new requirements coming along, but you need to do that in close work with the people maintaining the production one, to ensure that the way thinks evolve is actually something that can be mirrored in the area it is intended to be run later. (Alternatively you can have a free-for-all run development and then get a third env, in which you then try to get to run the new release in a (newer version of the) production env). Since DSA is using puppet extensively ATM, wouldn't it be better to have a documented procedure on how to set up a VM or chroot or similar environment that uses DSA's puppet recipes to set up a development instance? That way, people can make changes where necessary (while obviously understanding these changes may or may not be acceptable), don't have to worry about making a mistake and killing someone else's machine (after all, it's their own machine), etc. Just a thought. -- This end should point toward the ground if you want to go to space. If it starts pointing toward space you are having a bad problem and you will not go to space today. -- http://xkcd.com/1133/ -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52cbbf2d.2040...@debian.org
Re: Debian services and Debian infrastructure
On Jan 7, 2014 3:48 AM, Wouter Verhelst wou...@debian.org wrote: Op 05-01-14 14:28, Joerg Jaspert schreef: On 13446 March 1977, Lucas Nussbaum wrote: I'm worried that this situation is harmful for the project. Yes, seperate development with different requirements than the production environment is harmful. It's a lesson that every larger company with an IT Department goes through at some point - and it's the same for our project. You need a development environment that is build and handled the same way as the production environment. Sure you need to evolve that, it may not stand still, with new requirements coming along, but you need to do that in close work with the people maintaining the production one, to ensure that the way thinks evolve is actually something that can be mirrored in the area it is intended to be run later. (Alternatively you can have a free-for-all run development and then get a third env, in which you then try to get to run the new release in a (newer version of the) production env). Since DSA is using puppet extensively ATM, wouldn't it be better to have a documented procedure on how to set up a VM or chroot or similar environment that uses DSA's puppet recipes to set up a development instance? That way, people can make changes where necessary (while obviously understanding these changes may or may not be acceptable), don't have to worry about making a mistake and killing someone else's machine (after all, it's their own machine), etc. Just a thought. Well the toolset that people in the fast moving startup world are using to provide devs with a local dev environment that very much approximates their multinode prod envs is a combination of vagrant, virtualbox, and the puppet/chef code used to provision those environments. (Basically a toolset to automate the provisioning of local VMs that are configured like their production machines). I don't know enough about DSA infrastructure or the issues to say whether or not this would work for us, but it may be worth investigating. -Brian -- This end should point toward the ground if you want to go to space. If it starts pointing toward space you are having a bad problem and you will not go to space today. -- http://xkcd.com/1133/ -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52cbbf2d.2040...@debian.org
Re: Debian services and Debian infrastructure
On Tue, Jan 07, 2014 at 08:32:23AM -0500, Brian Gupta wrote: Well the toolset that people in the fast moving startup world are using to provide devs with a local dev environment that very much approximates their multinode prod envs is a combination of vagrant, virtualbox, and the puppet/ chef code used to provision those environments. (Basically a toolset to automate the provisioning of local VMs that are configured like their production machines). I don't know enough about DSA infrastructure or the issues to say whether or not this would work for us, but it may be worth investigating. Our production environments are debian stable+backports, and all web application dependencies should be debian packages from stable+backports. I don't think we need any fast moving startup infrastructure: just debootstrap, puppet to configure the VM (or a copy of the server's apache configuration), and apt-get. We are a distribution, and already we do very good integration work. Ciao, Enrico -- GPG key: 4096R/E7AD5568 2009-05-08 Enrico Zini enr...@enricozini.org signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
This one time, at band camp, Lucas Nussbaum said: Hi, As previously mentioned[1], I've been talking with DSA about the state of Debian services on Debian infrastructure. DSA asked me to move the discussion to a public list, which is what I'm doing here. What follows is my current state of mind on this question. I'm open to changing my mind based on the project's feedback. [1] https://lists.debian.org/debian-project/2013/12/msg00015.html The underlying questions that I'd like to answer are: Q1. How much should we push for Debian services (services useful to Debian) to be hosted on Debian infrastructure? Q2. Should we seek other hosting opportunities to ease Debian services development and hosting? Should I authorize the use of Debian money to fund infrastructure not managed by DSA, in the case of a useful service that DSA has been unable to accept in the infrastructure it manages? Q3. What should be our definition of official services? Background == First, it's important to state that DSA has been doing a fantastic job on maintaining our core infrastructure for the last few years. The level of quality and professionnalism of their work clearly surpasses what can be found in most organizations, volunteer or not. Thank you. I am speaking largely for myself, but I've discussed this mail with the team. Where I use 'we', I speak for the team. However, there has been several episodes, involving the development of new services or their move to Debian infrastructure, that generated a lot of frustration and demotivation on both sides (DSA and service developers). As examples, one can think of codesearch, or the fedmsg GSOC project. There are also services that have been developed outside of Debian's infrastructure, with AFAIK no plans to move them to Debian infrastructure. The recurring pattern seems to be that, when DSA is approached to move the service to Debian infrastructure, their evaluation of the service's design results in changes requests or constraints that the service maintainers consider too hard to satisfy. We think this might come from a fundamental mismatch between what DSA are willing to host and what some people would like to design. We don't think that that's a bad thing: we don't have to host everything that relates to Debian, any more than the glibc maintainers have to look after everything written in C. We are, of course, happy to look after things once they get a little further along and look like they'll prove useful to the project. I'm worried that this situation is harmful for the project. Similarly to how we increased the amount of collaborative packages maintainance over the years, we should improve on collaborative service maintainance, and we should move away from the myriad of unofficial services maintained by a sole DD, and hosted on this DD's personal machine. Absolutely agreed. The obvious way to do that would be to facilitate the hosting of emerging services on Debian infrastructure, and I think that we should all work together to identify what could be done towards that. We don't agree here. There are many ways to meet the goal of service ownership being collaborative, only one of which is to have everything hosted by DSA. Another might be that service owners look after their own hosting and system administration duties themselves, in a team, and have nothing to do with DSA. Another might be that DSA act in an advisory capacity, and assist new services in starting up, and then recuse ourselves and allow the service owners to look after their service themselves. Another might be that there is a mixed mode of support. I don't think jumping straight to a solution that puts all of the responsibility for every idea for a service in Debian on DSA shoulders is either the only way to go or even a good way to go. There are plenty of bad ideas that should be allowed to wither on the vine, and there are always going to be services that have been designed in such a way as to be difficult to integrate into DSA-managed infrastructure. We are, after all, a reasonably small team of volunteers. Pretending that we can support an infinite number of services or an infinite variety of designs is just going to end in disappointment for someone. I've given some thought to this myself, and came up with the following ideas. Some of them are probably really bad ideas, but let's try to brainstorm a bit: 1. to improve communication between DSA and service maintainers + have an archived, public list where (prospective) service maintainers can engage with DSA about stuff they are planning/thinking about. (Maybe recycle debian-admin@l.d.o, or use debian-services-admin@l.d.o?) + have DSA provide collective answers more often, rather than a set of individual answers. It's often not clear if something is a final decision, or the opinion of a DSA member. + redirect some
Re: Debian services and Debian infrastructure
On Mon, January 6, 2014 11:38, Stephen Gran wrote: I don't think jumping straight to a solution that puts all of the responsibility for every idea for a service in Debian on DSA shoulders is either the only way to go or even a good way to go. There are plenty of bad ideas that should be allowed to wither on the vine, and there are always going to be services that have been designed in such a way as to be difficult to integrate into DSA-managed infrastructure. We are, after all, a reasonably small team of volunteers. Pretending that we can support an infinite number of services or an infinite variety of designs is just going to end in disappointment for someone. I think this is a good observation. The whole problem does not seem to differ at all from with the tradeoffs that the sysadmin team in a large organisation wherein I work, have to make regularly. New services come to us in a variety of forms: someone just requests functionality and all other decisions are left to us; someone wants us to host a product or is developing a product and consults us early on (those people are the best!); more often someone has already bought or developed some product and it 'just' needs to be hosted. It's always a case by case judgement: how essential is this service? Will this fit in our infrastructure? If not, can we feasibly have it changed so it will? And if not, sometimes it's decided that we will need to host it nonetheless, in other cases we advise to host it somewhere else. The latter is surely an option and it doesn't necessarily mean we completely lose control over it at all. The simple fact that we retain control over DNS is a blunt but effective last resort to anything gone bad. Just like DSA, it's not our job to exclusively host everything that the organisation requires nor is it required that everything that we do end up hosting conforms to our ideal of the perfect service. Cheers, Thijs -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/f1fda1389bd5e7727d4294a794afe446.squir...@aphrodite.kinkhorst.nl
Re: Debian services and Debian infrastructure
Hi. Enrico Zini enr...@enricozini.org writes: On Sun, Jan 05, 2014 at 12:05:18PM +0100, Raphael Hertzog wrote: 1. to improve communication between DSA and service maintainers + have an archived, public list where (prospective) service maintainers can engage with DSA about stuff they are planning/thinking about. (Maybe recycle debian-admin@l.d.o, or use debian-services-admin@l.d.o?) debian-services-admin@l.d.o seems to be rather appropriate for this (it's unlikely that this kind of discussion would generate so much mail that the list could no longer serve its current purpose). debian-services-admin@l.d.o seems to me, too, to be the perfect place for it. +1 This would be good to have but (someone|DSA) would have to prod service maintainers to get this started, otherwise it will never happen. This requirement could also be mentioned in the document I suggested earlier. I think we already have something very close to this description here: https://wiki.debian.org/Services +1 Both the list and the wiki pages some love from DSA or other project members, IMHO. I've tried to ask for information and document the bits I could get, on recent experiments in the domain of such services, and found it really hard with very low (none) feedback on my contributions there :-/ Thanks Lucas for bringing this topic to discussion, and hoping the situation improves. My 2 cents, -- Olivier BERGER http://www-public.telecom-sudparis.eu/~berger_o/ - OpenPGP-Id: 2048R/5819D7E8 Ingenieur Recherche - Dept INF Institut Mines-Telecom, Telecom SudParis, Evry (France) -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87txdhm47i@inf-8660.int-evry.fr
Re: Debian services and Debian infrastructure
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 05/01/14 14:55, Stefano Zacchiroli wrote: On Sun, Jan 05, 2014 at 02:28:59PM +0100, Joerg Jaspert wrote: Q3. What should be our definition of official services? Even if this is highly preferable, I don't think that official services (.d.o services) should necessarily be running on Debian hardware managed by DSA, provided that: - the service is clearly useful and used - the service has a sustainable maintainance model (active team + instructions on how to contribute, run a local copy, etc. + DFSG-free) - the service's design does not raise security or scalability concerns I disagree on that, official services should run under project overview. So far that the project can take it over and move it, should all of the team go away. Active team today doesn't mean they are there tomorrow to properly hand it over. Having the project itself have access (via DSA at least) ensures it can continue. I very much agree with Joerg on this. A team like DSA offers to the Debian Project two key features. One, which is the most visible one, is the technical output and continued service of a team of talented sysadmins. The other is the governance guarantee that we, as a project, can always take over the maintenance of a service whose current maintainers go MIA or become unwilling to work with the rest of the project for any reason. As Joerg points out. Disclaimer/conflict of interest: I'm currently proposing a lightweight SIP service to run on DSA maintained infrastructure For any service, there are some general issues that come to mind for me: a) credentials: many services need TLS certificates these days. debian.org private keys probably have to be on boxes that DSA control and secure and not floating around in shared cloud storage or outsourced boxes at arms length b) delegated services: that said, some types of application can delegate their security checks (e.g. SIP can use a RADIUS server to verify DIGESTs). In these situations, the service hosting equipment does not need to have any copy of the user credentials. The verifications are made in the RADIUS server. By exposing services such as RADIUS, DSA could allow other people to run servers that don't need any copies of keys or credentials on their local disks. c) purpose: we should define some criteria that make the service compelling. For example, DSA doesn't provide a full mailbox service, but a mail forwarding is available. Personally, I feel that it is compelling for Debian to offer some SIP and XMPP service on a similar level to mail forwarding, in particular, because of the strong position that proprietary alternatives in that space - but maybe that isn't a criteria that everybody else would value. What, then, should the list of criteria be for a service that we value or can't be without? -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/ iQIcBAEBCAAGBQJSyx0kAAoJEOm1uwJp1aqDk14P/R9HqlkhaLrjQ5oExZmj2M46 B6lVTKY3kVUL4dUamkhJ+IiLvl4zJ9ZV4sb4shxFEEPuKt95r18DdnB3EnzcDn7W SXcENMZgMyCrfsWmrLfjL7ignqpuOnJbJ1TLvVU6S5vcSHnmnrpVSxgZuqYRjevA ydq+6pM4mtaBiPkuOOKsp0YvQJUe2glTx3gW489ieNEu3Knzj1319g8/qqB7w2K0 mI8LN9nNP6d2Fxi5RwPS2XWSyIXwUc0FhXnBPjWbJVp0bPg7+39CJm4paXKNzk4W U79mC6u2q6A+ts5rfUVCoycfzfXK067KemRSKSzgZVg8B6FJ4Ez65kTQ5n8K6VNd hgFYRgXJ9jMpBlWOUAqeiDCMD/MxDKhTQdrhqd5fJO2Cwj/cbbCgZ7oplHha+T6W hOLgo6ib6iPs297EWl5hD3D9mG07ThCnJGqNtxBM3uESlcCzpBGlp3+DI/zTYALd 24OpswTDa/COIB23C9CwYc0omasSo4pWXm7H8wGoycsRmriYDRV3XOrDC6lltOqm Vik2SMs+qNBo3GG+BJl/C0s3BK2LgYpavaxecHBKUXeex1fxh5RESUoQmXzVXIZN QvKJl2T104wxVoKqsi1v9DacSV1Fh4eVmag2Ta9T7/KIYDtNAQSz9XKJ9/eOv9VM +ZYlb1Bc4jTg/9WOaaYk =/SaF -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52cb1d24.9090...@pocock.com.au
Re: Debian services and Debian infrastructure
On Mon, Jan 06, 2014 at 10:16:20PM +0100, Daniel Pocock wrote: On 05/01/14 14:55, Stefano Zacchiroli wrote: On Sun, Jan 05, 2014 at 02:28:59PM +0100, Joerg Jaspert wrote: Q3. What should be our definition of official services? Even if this is highly preferable, I don't think that official services (.d.o services) should necessarily be running on Debian hardware managed by DSA, provided that: - the service is clearly useful and used - the service has a sustainable maintainance model (active team + instructions on how to contribute, run a local copy, etc. + DFSG-free) - the service's design does not raise security or scalability concerns I disagree on that, official services should run under project overview. So far that the project can take it over and move it, should all of the team go away. Active team today doesn't mean they are there tomorrow to properly hand it over. Having the project itself have access (via DSA at least) ensures it can continue. I very much agree with Joerg on this. A team like DSA offers to the Debian Project two key features. One, which is the most visible one, is the technical output and continued service of a team of talented sysadmins. The other is the governance guarantee that we, as a project, can always take over the maintenance of a service whose current maintainers go MIA or become unwilling to work with the rest of the project for any reason. As Joerg points out. Disclaimer/conflict of interest: I'm currently proposing a lightweight SIP service to run on DSA maintained infrastructure For any service, there are some general issues that come to mind for me: a) credentials: many services need TLS certificates these days. debian.org private keys probably have to be on boxes that DSA control and secure and not floating around in shared cloud storage or outsourced boxes at arms length agreed b) delegated services: that said, some types of application can delegate their security checks (e.g. SIP can use a RADIUS server to verify DIGESTs). In these situations, the service hosting equipment does not need to have any copy of the user credentials. The verifications are made in the RADIUS server. By exposing services such as RADIUS, DSA could allow other people to run servers that don't need any copies of keys or credentials on their local disks. we're also concerned about which passwords are entered where; hence sso.debian.org and the desire to keep distinct the password used for SIP/XMPP from that used for sudo -- Luca Filipozzi http://www.crowdrise.com/SupportDebian signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
Hi, On Sat, 04 Jan 2014, Lucas Nussbaum wrote: I've given some thought to this myself, and came up with the following ideas. Some of them are probably really bad ideas, but let's try to brainstorm a bit: I don't find them bad. At least from the POV of view of a DD and of a service maintainer, they seem to go in the right direction. 1. to improve communication between DSA and service maintainers + have an archived, public list where (prospective) service maintainers can engage with DSA about stuff they are planning/thinking about. (Maybe recycle debian-admin@l.d.o, or use debian-services-admin@l.d.o?) debian-services-admin@l.d.o seems to be rather appropriate for this (it's unlikely that this kind of discussion would generate so much mail that the list could no longer serve its current purpose). + have DSA provide collective answers more often, rather than a set of individual answers. It's often not clear if something is a final decision, or the opinion of a DSA member. + redirect some discussions from IRC to a mailing list, esp. when things look like policy decisions (on service design, etc) It would also be helpful to have a document explaining their usual requirements. It would pro-actively direct the service maintainers towards choices that are acceptable to DSA. Things like: - use PostgreSQL if you need a DB - keep in mind that DSA likes to ensure high-availability, so make it easy to run a replicate of your service - avoid dependencies not in stable or stable-backports - avoid X, Y, Z for security reasons - etc. 2. to improve the tracking of services + request wiki pages from maintainers that detail who are the contact points, what the service does, what are its requirements. State a service level agreement (informative of expectations, not punitive, of course) + have service maintainers create similar pages for services in development, to provide some kind of incubation process during which DSA can provide feedback. This would be good to have but (someone|DSA) would have to prod service maintainers to get this started, otherwise it will never happen. This requirement could also be mentioned in the document I suggested earlier. 3. to provide a place to experiment with new services + create a Debian cloud with virtual machines to develop new services (maybe providing manually-created VMs would be enough -- I'm not sure we need a complex infra such as OpenStack). I think we're already close to this at least in terms of creating virtual machines for new services. I got one created rather easily and several others were added recently. That said a Debian cloud for short-lived Debian tasks is still an interesting ideas, possibly in relation with PPA and having the possibility to rebuild packages against a PPA. Q1. How much should we push for Debian services (services useful to Debian) to be hosted on Debian infrastructure? Fragmentation in terms of hosting is harmful, and we should all try to get our services hosted on Debian infrastructure, because that's the easiest way to ensure that the maintainance of such services can be shared or transferred between developers. Now, for some services, it seems that doing this has become so hard that it harmed the service itself, and badly demotivated the service maintainers. I don't think that we should allow that to happen. We should stay open to other hosting solutions, when this is necessary. Having stuff hosted on DSA-managed hardware seems like something important to me. That said we can't force all services to be ready to meet DSA's requirements from day 1. So my suggestion would be have DSA allocate VMs to host services in incubation and keep those services in the .debian.net umbrella until they are ready. There's always going to be exceptions (stuff needing so much resource that it's unrealistic to allocate them out of the current hardware pool) but that's OK. Those can be dealt on a case by case basis and are likely to benefit from early interaction with DSA. Q2. Should we seek other hosting opportunities to ease Debian services development and hosting? Should I authorize the use of Debian money to fund infrastructure not managed by DSA, in the case of a useful service that DSA has been unable to accept in the infrastructure it manages? For services development (something for which we don't have a good answer yet), I think that we should explore getting specific sponsored hosting. As you know, Amazon provides Debian with free credits on the AWS Cloud. Those credits are normally used for archive rebuilds, but have already been used by a GSOC project (PTS rewrite), and more recently, to start work on an archive-wide autopkgtest setup. A few more virtual machines could be accomodated, and I will investigate if we could extend that even more. Also, codesearch.d.n is hosted on Rackspace's
Re: Debian services and Debian infrastructure
On Sun, Jan 05, 2014 at 12:05:18PM +0100, Raphael Hertzog wrote: 1. to improve communication between DSA and service maintainers + have an archived, public list where (prospective) service maintainers can engage with DSA about stuff they are planning/thinking about. (Maybe recycle debian-admin@l.d.o, or use debian-services-admin@l.d.o?) debian-services-admin@l.d.o seems to be rather appropriate for this (it's unlikely that this kind of discussion would generate so much mail that the list could no longer serve its current purpose). debian-services-admin@l.d.o seems to me, too, to be the perfect place for it. It would also be helpful to have a document explaining their usual requirements. It would pro-actively direct the service maintainers towards choices that are acceptable to DSA. Things like: - use PostgreSQL if you need a DB - keep in mind that DSA likes to ensure high-availability, so make it easy to run a replicate of your service - avoid dependencies not in stable or stable-backports - avoid X, Y, Z for security reasons - etc. Also agreed. It sounds to me like a specifications of the production environment where things will be deployed, and it's something that as a developer I should see sooner rather than later. 2. to improve the tracking of services + request wiki pages from maintainers that detail who are the contact points, what the service does, what are its requirements. State a service level agreement (informative of expectations, not punitive, of course) + have service maintainers create similar pages for services in development, to provide some kind of incubation process during which DSA can provide feedback. This would be good to have but (someone|DSA) would have to prod service maintainers to get this started, otherwise it will never happen. This requirement could also be mentioned in the document I suggested earlier. I think we already have something very close to this description here: https://wiki.debian.org/Services Ciao, Enrico -- GPG key: 4096R/E7AD5568 2009-05-08 Enrico Zini enr...@enricozini.org signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
On 13446 March 1977, Lucas Nussbaum wrote: Q1. How much should we push for Debian services (services useful to Debian) to be hosted on Debian infrastructure? A lot. Should I authorize the use of Debian money to fund infrastructure not managed by DSA, in the case of a useful service that DSA has been unable to accept in the infrastructure it manages? Like wasting money on essentially dead systems, like m68k? However, there has been several episodes, involving the development of new services or their move to Debian infrastructure, that generated a lot of frustration and demotivation on both sides (DSA and service developers). As examples, one can think of codesearch, or the fedmsg GSOC project. From what I followed on some service, a good part of the frustration came from real unrealistic requirements that just got thrown in front of DSA. One thing i remember is Oh, we just need fast storage, a recent SSD is enough [...] its only a hundred or so dollars - which is an easy thing to say talking about desktop machines where you put one. But DSA operates on a different level and suddenly putting SSD storage came out multiple thousand dollars. Not surprising that it wasnt taken all that good a suggestion. And another can be summarized to here, take this, no, your ideas aren't considered, all the rest of the world eats this too, so you have to eat it also. Makes people love to help you. There are also services that have been developed outside of Debian's infrastructure, with AFAIK no plans to move them to Debian infrastructure. The recurring pattern seems to be that, when DSA is approached to move the service to Debian infrastructure, their evaluation of the service's design results in changes requests or constraints that the service maintainers consider too hard to satisfy. And that is a problem. Not directly with DSA, not directly with the service maintainers, but with the structure of having development of services done outside the infrastructure it later on should run on and then blindly assuming that the infrastructure can just take it. That does not work. I'm worried that this situation is harmful for the project. Yes, seperate development with different requirements than the production environment is harmful. It's a lesson that every larger company with an IT Department goes through at some point - and it's the same for our project. You need a development environment that is build and handled the same way as the production environment. Sure you need to evolve that, it may not stand still, with new requirements coming along, but you need to do that in close work with the people maintaining the production one, to ensure that the way thinks evolve is actually something that can be mirrored in the area it is intended to be run later. (Alternatively you can have a free-for-all run development and then get a third env, in which you then try to get to run the new release in a (newer version of the) production env). Similarly to how we increased the amount of collaborative packages maintainance over the years, we should improve on collaborative service maintainance, and we should move away from the myriad of unofficial services maintained by a sole DD, and hosted on this DD's personal machine. Oh yes, I don't think anyone will disagree with Services for/around Debian should be run on official hosted resources. 1. to improve communication between DSA and service maintainers + have an archived, public list where (prospective) service maintainers can engage with DSA about stuff they are planning/thinking about. (Maybe recycle debian-admin@l.d.o, or use debian-services-admin@l.d.o?) + Have service maintainers accept that other people (eg DSA) have knowledge too. And accept their input. + have DSA provide collective answers more often, rather than a set of individual answers. It's often not clear if something is a final decision, or the opinion of a DSA member. If it gets a requirement that DSA has to give collective answers and otherwise should refrain from commenting, then its bad. Also, an individual DSA answering and giving his opinion shouldn't be a problem - it only would if the team itself is splitted and you get 5 opinions from 3 people. 2. to improve the tracking of services + request wiki pages from maintainers that detail who are the contact points, what the service does, what are its requirements. State a service level agreement (informative of expectations, not punitive, of course) + have service maintainers create similar pages for services in development, to provide some kind of incubation process during which DSA can provide feedback. + have service maintainers engage with DSA *early*, as early as remotely possible, listen to them and work with them to get useful (for both sides) compromises to make the service a good guest on Debian
Re: Debian services and Debian infrastructure
On Sun, Jan 05, 2014 at 02:28:59PM +0100, Joerg Jaspert wrote: Q3. What should be our definition of official services? Even if this is highly preferable, I don't think that official services (.d.o services) should necessarily be running on Debian hardware managed by DSA, provided that: - the service is clearly useful and used - the service has a sustainable maintainance model (active team + instructions on how to contribute, run a local copy, etc. + DFSG-free) - the service's design does not raise security or scalability concerns I disagree on that, official services should run under project overview. So far that the project can take it over and move it, should all of the team go away. Active team today doesn't mean they are there tomorrow to properly hand it over. Having the project itself have access (via DSA at least) ensures it can continue. I very much agree with Joerg on this. A team like DSA offers to the Debian Project two key features. One, which is the most visible one, is the technical output and continued service of a team of talented sysadmins. The other is the governance guarantee that we, as a project, can always take over the maintenance of a service whose current maintainers go MIA or become unwilling to work with the rest of the project for any reason. As Joerg points out. I understand that we can come up with a list of requirements that diminish the risk. This is, I believe, what Lucas was trying to do with the requirements quoted above. But I think it is much better to centralize those requirements on a single, DPL-delegated team, than trying to replicate that to many different, de facto self-proclaimed teams. Cheers. -- Stefano Zacchiroli . . . . . . . z...@upsilon.cc . . . . o . . . o . o Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o Former Debian Project Leader . . @zack on identi.ca . . o o o . . . o . « the first rule of tautology club is the first rule of tautology club » signature.asc Description: Digital signature
Re: Debian services and Debian infrastructure
We've implemented OpenNebula here at Brainfood and after having had some experience with it I would say that having that kind of infrastructure simplifies management rather than making it more complicated. Having a bunch of hand baked KVMs running without coordinated migration or shared storage is a lot more fragile, confusing and slow to work with. For instance, if we need to service a machine we can tell OpenNebula to flush that host and it will automatically balance all the virtual machines running on that machine to other hosts. That's a big convenience. I'm sure OpenStack has similar capabilities. We evaluated it along with a number of other infrastructures and for us OpenNebula was just the easiest to get going from the stock Debian packages. We also have found it to be very hackable. A lot of its functionality actually lives in shell scripts rather than some DSL or Ruby code. - Lucas Nussbaum lea...@debian.org wrote: 3. to provide a place to experiment with new services + create a Debian cloud with virtual machines to develop new services (maybe providing manually-created VMs would be enough -- I'm not sure we need a complex infra such as OpenStack). -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/27143708.9661388951071865.javamail.r...@newmail.brainfood.com
Re: Debian services and Debian infrastructure
On 13447 March 1977, Ean Schuessler wrote: 3. to provide a place to experiment with new services + create a Debian cloud with virtual machines to develop new services (maybe providing manually-created VMs would be enough -- I'm not sure we need a complex infra such as OpenStack). We've implemented OpenNebula here at Brainfood and after having had some [opennebula usage description] Note that DSA is not manually managing each little VM. They do use Ganeti. I'm not up comparing that to any cloud stuff, but it might do enough for what is suggested to have. -- bye, Joerg Some NM: Essential: Yes -- useful for a message when you do apt-get remove bash: -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87vbxym9lk@gkar.ganneff.de
