Re: Help bringing bugs.debian.org / debbugs back on track (Re: bits from the DPL -- September 2013)
On Wed, Oct 16, 2013 at 09:38:31AM +0800, Paul Wise wrote: Please don't switch bugs.debian.org away from debbugs. I don't want to have to leave the Debian project but some misguided folks doing that would be one of the triggers for that. Please don't threaten/ransom your labour to push one or another side of a debate. -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131016094024.ga11...@bryant.redmars.org
Re: Help bringing bugs.debian.org / debbugs back on track (Re: bits from the DPL -- September 2013)
On Tue, Oct 15, 2013 at 08:51:15PM -0400, Filipus Klutiero wrote: Thanks for bringing this up, our ITS definitely needs love. I investigated the ITS team in 2010 and found it was already broken (although one member disagreed with that assessment, qualifying the team's status as fine). The most urgent issue must be fixing the administration of bugs.debian.org, but development should also be a priority. To set the record straight for anybody fortunate enough to not have any background here: Filipus / Philippe / chealer (to pick some of his names) has been a thorn in the side of the BTS admins for several years. My own experience from talking with him and the BTS folks is that he's very difficult to deal with. His broken comment above undoubtedly refers to his experiences when he was banned from using a lot of the BTS features for abuse. As such, if there is anybody whose opinions should be ignored more about how our bug tracker should be run and developed, I would struggle to identify such a person. Thank you for reading. -- Steve McIntyre, Cambridge, UK.st...@einval.com I can't ever sleep on planes ... call it irrational if you like, but I'm afraid I'll miss my stop -- Vivek Dasmohapatra -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131016095443.gx14...@einval.com
Re: Possibly moving Debian services to a CDN
On Sun, Oct 13, 2013 at 08:44:56AM +0200, Tollef Fog Heen wrote: We appreciate feedback while we continue our investigation of CDNs. Hi Tollef, thanks for bringing this discussion to -project. I'm myself against switching to a CDN, but it might be due to a lack of information on my part, so I'd love if DSA could fill in my gaps. Debian is a Free Software project. I think that is what should drive our choices and nothing else. As such I'd hate seeing Debian moving, by default, to a content delivery solution that is made of proprietary software parts. That would be very bad for the Debian Project, as it would send the message that while we do create an entirely Free OS for others, we are fine with using proprietary software to do so (Mako has expressed this concept much better than I could possibly do in his Free Software Needs Free Tools essay http://mako.cc/writing/hill-free_tools.html ). In my mind, using a CDN made of non-free parts would be exactly the same as using a proprietary BTS, or replacing the DBMS that backs dak with Oracle Database. I do realize that most of the value of a CDN is not in its software parts. But I'm under the impression there is still quite a bit of software behind commercial CDN offerings. So my question is: would the CDN providers we're going to choose be able to ensure that the software parts behind their offerings to us are 100% Free Software? I don't think we have enough leverage to impose that. But if it is the case my non 100% Free Software concern above would certainly disappear. Another way of making my concern moot would be to use the CDN only as a non-default option that users should explicitly choose, and label it as some non official service. That would reduce the impression that the Debian Project endorses infrastructures that rely on non-free software. For instance, we could repurpose the cdn.debian.net name (assuming the current maintainer is fine with the idea) and present it as an option to our users. A corollary of this is that it would be difficult to use the CDN for things like www.debian.org (probably making moot many of the advantages you're looking for). An unrelated concern is that of technical independence. I do see a lot of value in Debian social experiment (quoting here a very nice way of calling it that Lucas has come up to) of trying to do almost everything by ourselves, from packaging to legal, from marketing to sysadm'ing. Of course it comes with a lot of drawbacks, and I don't think it is something rooted in Debian principles. But it is a very nice characteristic of our Project, and I think we should be very careful before giving it up, even if in only in specific areas. In your mail you've addressed the concern of excessive dependence on a *single* CDN provider, mentioning that you're looking into how easily switch from one provider to another. Would it be equally easy to get rid of the CDN --- or switch to a more home brew (set of) CDN(s) --- if things go awry? (FWIW, I'm not myself worried about dependency on money / hw coming from companies, I'm very well aware that Debian needs quite a bit of resources from companies. But that concern is very much mitigated by the diversity in donors, or at least by the fact that we can have that diversity. Technical dependency is IMHO much worse, because to get diversity there you need to have in place, beforehand, the needed abstraction layers.) Cheers. -- Stefano Zacchiroli . . . . . . . z...@upsilon.cc . . . . o . . . o . o Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o Former Debian Project Leader . . @zack on identi.ca . . o o o . . . o . « the first rule of tautology club is the first rule of tautology club » signature.asc Description: Digital signature
Re: Possibly moving Debian services to a CDN
On Wed, October 16, 2013 15:01, Stefano Zacchiroli wrote: I do realize that most of the value of a CDN is not in its software parts. But I'm under the impression there is still quite a bit of software behind commercial CDN offerings. So my question is: would the CDN providers we're going to choose be able to ensure that the software parts behind their offerings to us are 100% Free Software? It is of course a good target to strive for, but putting such a demand on a service is probably not fair, because our current distribution system also does not run on 100% Free Software. We do not know what software our mirrors are running (we can make a guess that there will be Debian hosts in there, but just as well we can be quite sure there will be some Solaris or commercial Linux variants) so it's just as opaque as a commercial CDN offering would be. The principled point that everything Debian does should involve 100% Free Software is not the status quo nor is it attainable. We will deliver bits using equipment running Cisco IOS, and we will not have the source of the bank software that processes our donations. The question is therefore: do we consider such CDN services to be more like network- or financial services, which we source externally, or is it a core aspect of developing Debian? Cheers, Thijs -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/459f0e787bedca53b8a40256c4050c1f.squir...@aphrodite.kinkhorst.nl
Re: Possibly moving Debian services to a CDN
On Wed, Oct 16, 2013 at 03:40:48PM +0200, Thijs Kinkhorst wrote: It is of course a good target to strive for, but putting such a demand on a service is probably not fair, because our current distribution system also does not run on 100% Free Software. We do not know what software our mirrors are running (we can make a guess that there will be Debian hosts in there, but just as well we can be quite sure there will be some Solaris or commercial Linux variants) so it's just as opaque as a commercial CDN offering would be. I disagree with that, both on a principle basis and on a practical one. Right now, as a project, we do not take any stance on the free-ness of our mirror network. It's some sort of don't ask, don't tell regime. In practical terms I'm myself convinced that many mirrors are run entirely on Free Software (see about network equipment below), and I can guarantee that was the case for mirrors I've been running myself in the past. Switching *by default* to a CDN could make the situation much better (if the offer is 100% Free Software, which is unfortunately unlikely) or much worse (the most likely case). If the situation is going to get much worse, I frankly prefer the status quo. BTW, I forgot to add a very important note to the previous mail. I realize that switching to a CDN could make things much better for DSA, and that's why it pains me to object to the idea by being someone that is not contributing in anyway to the DSA team. I do that only because I think we're here on a topic where what I believe to be Debian principles are at stake. The principled point that everything Debian does should involve 100% Free Software is not the status quo nor is it attainable. We will deliver bits using equipment running Cisco IOS, and we will not have the source of the bank software that processes our donations. The question is therefore: do we consider such CDN services to be more like network- or financial services, which we source externally, or is it a core aspect of developing Debian? Right, that's indeed the question. And I realize that different people will place their bars at different levels. My bar is that content delivery is indeed a core aspect for a project producing a Free Software distribution. YMMV, and I've no desire of trying to convince otherwise people who place their bars differently. But this thread was about opinions on the idea; this is mine :-) -- Stefano Zacchiroli . . . . . . . z...@upsilon.cc . . . . o . . . o . o Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o Former Debian Project Leader . . @zack on identi.ca . . o o o . . . o . « the first rule of tautology club is the first rule of tautology club » signature.asc Description: Digital signature
Re: Possibly moving Debian services to a CDN
Hi, (cc debian-mirr...@lists.debian.org, the list where such discussions can happen) On Sun, Oct 13, 2013 at 09:00:04PM -0700, Russ Allbery wrote: Joey Hess jo...@debian.org writes: But apparently not one solved by free software included in Debian. Perhaps it's worth avoiding using it if that will help encourage the development of libre alternatives. CDNs aren't really software problems. They're infrastructure and network peering problems. I think all the software required is in Debian, but the data centers, peering arragements, route advertisements, and so forth are things that only make sense to do at a larger scale than a single project. We already have a network of almost 400 packages mirrors around the world. Using http.d.n, provides the CDN layer (not as much as optimal as anycast), so we don't need to sort ourselves peering issues etc. We can do a home-brewed CDN -- that, after all, is what the various services referenced in the original message are. But one of the commercial CDNs will have better performance and better load distribution than one can do with software-only solutions without the peering setup and data center distribution. * Commercial CDN have no knowledge of debian archive datamodel and constraints, which leads to inconsistency (and consequently hash sum dismatch). Or this would imply just disabling caching for problematic files, or change apt/dak so that the archive is less impacted by synchro atomicity issues. * My own experience is different, http.d.n redirects to ftp2.fr, which i got 10,2Mo/s, while cloudfront.d.n (Amazon) gives 5Mo/s. -- Simon Paillard mirr...@debian.org team member -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131016182336.gq5...@mraw.org
Re: Possibly moving Debian services to a CDN
This one time, at band camp, Simon Paillard said: Hi, We already have a network of almost 400 packages mirrors around the world. Using http.d.n, provides the CDN layer (not as much as optimal as anycast), so we don't need to sort ourselves peering issues etc. The mirrors do a very good job of being near to our users in most cases (we have had some difficulty with things like security mirror coverage, but that's not your issue). I know many people are happy with http.d.n, but you have to understand that it's not the same thing at all as a CDN - it's a single host, and it's in a single location. Time to first byte will still suffer if you're coming from Australia. We can do a home-brewed CDN -- that, after all, is what the various services referenced in the original message are. But one of the commercial CDNs will have better performance and better load distribution than one can do with software-only solutions without the peering setup and data center distribution. * Commercial CDN have no knowledge of debian archive datamodel and constraints, which leads to inconsistency (and consequently hash sum dismatch). Or this would imply just disabling caching for problematic files, or change apt/dak so that the archive is less impacted by synchro atomicity issues. Or do things like set long cache time on all files, and invalidate the cache on the metadata files once sync is complete. There are ways to solve those issues, just as there are ways to solve them when we're transferring them with rsync. * My own experience is different, http.d.n redirects to ftp2.fr, which i got 10,2Mo/s, while cloudfront.d.n (Amazon) gives 5Mo/s. This is an important data point. If cloudfront is outperformed by our current mirror infrastructure, it becomes less appealing. After all, we're trying to make things better, not worse. Cheers, -- - | ,''`.Stephen Gran | | : :' :sg...@debian.org | | `. `'Debian user, admin, and developer | |`- http://www.debian.org | - signature.asc Description: Digital signature
Re: Possibly moving Debian services to a CDN
Hi, On Wed, Oct 16, 2013 at 07:54:04PM +0100, Stephen Gran wrote: This one time, at band camp, Simon Paillard said: We already have a network of almost 400 packages mirrors around the world. Using http.d.n, provides the CDN layer (not as much as optimal as anycast), so we don't need to sort ourselves peering issues etc. The mirrors do a very good job of being near to our users in most cases (we have had some difficulty with things like security mirror coverage, but that's not your issue). I know many people are happy with http.d.n, but you have to understand that it's not the same thing at all as a CDN - it's a single host, and it's in a single location. Time to first byte will still suffer if you're coming from Australia. Obviously, it's due to current unsponsored deployment. * For a package of several kB, the first bytes (actually ~800B, measured) are not meaningful * Our (mirrors@d.o) is to have http.d.n made official so that we can have an instance like on each continent, then use GeoDNS, like security.d.o mirrors, to achieve 2 goals: avoid SPOF, be closer to users. -- Simon Paillard mirrors team member -- To UNSUBSCRIBE, email to debian-project-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131016190108.gt5...@mraw.org
