Re: Source packages appropriate by default?
On Tue, Jul 23, 2013 at 10:59:41PM -0400, Scott Kitterman wrote: Has any work been done on concurrent requests? That would likely be pretty broadly useful, not just for cloud images. AIUI, apt-get already supports concurrent requests, but only to diffferent servers at once. From my understanding of the code, queueing in a way to handle multiple concurrent requests to the same server has some internal API but isn't implemented. I believe that S3 is higher latency than normal as a consequence of its higher resiliency, and that you're supposed to use parallelism to improve performance. I do hope to enhance apt-get, but it's quite involved so haven't found enough time for it yet. -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On Tue, Jul 23, 2013 at 09:31:15PM -0400, Scott Kitterman wrote: /ubuntu/dists/raring-security/main/source [ ] Release 24-Jul-2013 01:16 106 [ ] Sources.bz2 24-Jul-2013 01:16 32K [ ] Sources.gz 24-Jul-2013 01:16 38K For end users, how much is really downloaded? 3 x 900 ms == 2.7 seconds /ubuntu/dists/raring-updates/main/source [ ] Release 24-Jul-2013 01:16 105 [ ] Sources.bz2 24-Jul-2013 01:16 50K [ ] Sources.gz 24-Jul-2013 01:16 62K 3 x 900 ms == 2.7 seconds /ubuntu/dists/raring-updates/universe/source [ ] Release 24-Jul-2013 01:16 109 [ ] Sources.bz2 24-Jul-2013 01:16 64K [ ] Sources.gz 24-Jul-2013 01:16 77K It doesn't seem like a lot. 3 x 900 ms == 2.7 seconds 2.7 x 3 == 8.1 seconds. And yes, I've seen 900ms latency to our servers before, and I'm on a reasonably quick connection, even by Korean or Swedish standards. I'd hate to think what connectivity to our servers would be if I were on a poor connection. (I'm far happier now that I've switched to mirror.anl.gov; 70 ms latency means my apt-get updates are now done in just 47 seconds rather than the nearly two minutes I got when using our servers. But my needs are a bit of an outlier..) Perhaps HTTP/2's better use of single sessions[1] will improve things drastically, but waiting for HTTP/2-compatible apt-get and servers feels like a long way to go around turning off the deb-src lines by default. [1]: http://www.chmod777self.com/2013/07/http2-status-update.html Thanks signature.asc Description: Digital signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On Tue, Jul 23, 2013 at 10:59:41PM -0400, Scott Kitterman wrote: Has any work been done on concurrent requests? That would likely be pretty broadly useful, not just for cloud images. AIUI, apt-get already supports concurrent requests, but only to diffferent servers at once. From my understanding of the code, queueing in a way to handle multiple concurrent requests to the same server has some internal API but isn't implemented. I believe that S3 is higher latency than normal as a consequence of its higher resiliency, and that you're supposed to use parallelism to improve performance. I do hope to enhance apt-get, but it's quite involved so haven't found enough time for it yet. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Tuesday, July 23, 2013 06:59:43 AM Robie Basak wrote: On Tue, Jul 23, 2013 at 01:51:46AM -0400, Scott Kitterman wrote: I think most developers would believe the current situation is appropriate. I disagree. By default users have the same access to source and binary packages and for a free software distribution, that is the ethically correct approach. Indeed, but you never replied to my original response to your concern. By same access, do you specifically require the mechanism to be to keep users' local apt caches maintained with source entries? If so, why is such a mechanism necessary to fit the spirit of Free Software? If the user still has easy access to the source, why is this not sufficient? I'm happy to discuss what easy access might actually mean, but I see no reason that it should require the waste of users' bandwidth and time. Sorry. I didn't mean to ignore you. What's easy? For example, I think install more packages to get the tools to get the source (use pull-lp-source in ubuntu-dev-tools) doesn't qualify. There are tons of documentation all over the web and other places as well that assume apt-get source works. I think access using installed tools that are normally used for the job (wget is installed (I think) by default, but I don't think having to go to a web page to find a URL and then wget'ing the components of the source package is easy either. So those are a couple of examples of what I think is definitely not what we want. I'm open to discussion about alternate ways to preserve easy access to the source. Scott K -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
(pardon the top-posting) I think the slight reduction in ethics (relevant mainly to developers) is a good trade to help deployability in the real world. We'll leave sources enabled by default for development releases. For the other 99% of users, where practicality is more important than immediate access to source, we end up wasting ~10% of Canonical and our mirror's bandwidth on the source updates. This makes a difference when behind a congested network, running on battery or so on. That 10% when accessing security.ubuntu.com really helps, particularly when topologically distant from the UK (if you have good network connectivity, ask someone who hasn't got it). No? On 23 July 2013 13:51, Scott Kitterman ubu...@kitterman.com wrote: On Tuesday, July 23, 2013 11:02:00 AM Daniel J Blueman wrote: By large, developers are uninterested in this, but it is important for users and where we use Ubuntu. Anyone care to comment on how we can progress this? I think most developers would believe the current situation is appropriate. By default users have the same access to source and binary packages and for a free software distribution, that is the ethically correct approach. Scott K On 15 July 2013 13:32, Daniel J Blueman dan...@quora.org wrote: From earlier feedback, there were no overriding reasons why package sources should be enabled by default. We not only save congestion on security.ubuntu.com, but quite a lot of country-level mirrors point to Canonical's servers, which are relatively distant and slow (~80KB/s from here), so this is a win. So, what's the path to change this? On 21 May 2013 22:04, J Fernyhough j.fernyho...@gmail.com wrote: On 21 May 2013 13:55, Robie Basak robie.ba...@canonical.com wrote: What if we provided a reasonable message if no deb-src lines are defined, with a single simple command to add them and run apt-get update for you? I don't think it would even need that - software-properties (Software Updates) already has the necessary checkbox. All that is needed to enable sources is to tick that box. From a technical point of view, does mirroring the deb lines into deb-src lines work in all cases? Would doing so break anything? This is effectively what Software Sources does under-the-hood. I have to agree, if the amount being downloaded is not trivial (which I thought it was) then there's no need to have them enabled by default when it's very easy to turn them on. One of the first things I do on any new install is disable those that aren't needed. Jonathon (to the list this time) -- Ubuntu-devel-discuss mailing list ubuntu-devel-disc...@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss -- Daniel J Blueman -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel -- Daniel J Blueman -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On Tue, Jul 23, 2013 at 01:51:46AM -0400, Scott Kitterman wrote: I think most developers would believe the current situation is appropriate. I disagree. By default users have the same access to source and binary packages and for a free software distribution, that is the ethically correct approach. Indeed, but you never replied to my original response to your concern. By same access, do you specifically require the mechanism to be to keep users' local apt caches maintained with source entries? If so, why is such a mechanism necessary to fit the spirit of Free Software? If the user still has easy access to the source, why is this not sufficient? I'm happy to discuss what easy access might actually mean, but I see no reason that it should require the waste of users' bandwidth and time. Robie -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On Tue, Jul 23, 2013 at 03:02:02AM -0400, Scott Kitterman wrote: So those are a couple of examples of what I think is definitely not what we want. I'm open to discussion about alternate ways to preserve easy access to the source. How about: $ sudo apt-get source hello Reading package lists... Done Building dependency tree Reading state information... Done E: You must put some 'source' URIs in your sources.list E: Type add-apt-repository sources to do this automatically for you. $ sudo add-apt-repository sources deb-src lines have been added to your sources.list. Now type apt-get update, and then apt-get source ... will work. $ sudo apt-get update (...) $ sudo apt-get source hello (works) To do this, we'd need to patch apt to add the second error line, and implement sources to add-apt-repository. Robie -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On 23.07.2013 09:12, Robie Basak wrote: [...] E: You must put some 'source' URIs in your sources.list E: Type add-apt-repository sources to do this automatically for you. $ sudo add-apt-repository sources deb-src lines have been added to your sources.list. Now type apt-get update, and then apt-get source ... will work. $ sudo apt-get update (...) $ sudo apt-get source hello (works) To do this, we'd need to patch apt to add the second error line, and implement sources to add-apt-repository. andreas@j3515:~$ sudo add-apt-repository The program 'add-apt-repository' is currently not installed. You can install it by typing: sudo apt-get install python-software-properties andreas@j3515:~$ add-apt-repository is not available on all Ubuntu systems by default. The current solution is easy and is available on virtually all installations, without the need to manually install anything. Providing easy access to the source is essential for a open source distribution. signature.asc Description: OpenPGP digital signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
Hi Daniel (2013.07.23_08:13:47_+0200) For the other 99% of users, where practicality is more important than immediate access to source, we end up wasting ~10% of Canonical and our mirror's bandwidth on the source updates. Can you back that up with evidence? As I (and a few other people) have repeatedly said in this thread: The release pocket lists aren't changed after release. Only -updates, -security, -backports and -proposed change, and they are all small because they are an overlay on the release pocket. SR -- Stefano Rivera http://tumbleweed.org.za/ H: +27 21 461 1230 C: +27 72 419 8559 -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On 23 July 2013 16:24, Andreas Moog andreas.m...@warperbbs.de wrote: On 23.07.2013 09:12, Robie Basak wrote: [...] E: You must put some 'source' URIs in your sources.list E: Type add-apt-repository sources to do this automatically for you. $ sudo add-apt-repository sources deb-src lines have been added to your sources.list. Now type apt-get update, and then apt-get source ... will work. $ sudo apt-get update (...) $ sudo apt-get source hello (works) To do this, we'd need to patch apt to add the second error line, and implement sources to add-apt-repository. andreas@j3515:~$ sudo add-apt-repository The program 'add-apt-repository' is currently not installed. You can install it by typing: sudo apt-get install python-software-properties andreas@j3515:~$ add-apt-repository is not available on all Ubuntu systems by default. add-apt-repository is installed by default on desktop installs. On non-desktop seeds, it is still available, just not installed. The current solution is easy and is available on virtually all installations, without the need to manually install anything. Providing easy access to the source is essential for a open source distribution. You're focussing on Ubuntu as a complete solution and not a platform or enabler in stating that easy access to source is essential. Do you agree that 99% of users don't care about source? For those who do, it's easy to find and tweak sources button in the software sources applet, no? From deploying in the real world (eg countries without domestic mirrors or with radio links), reducing bandwidth requirements of updates makes a important difference. Daniel -- Daniel J Blueman -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On Tue, Jul 23, 2013 at 10:24:31AM +0200, Andreas Moog wrote: andreas@j3515:~$ sudo add-apt-repository The program 'add-apt-repository' is currently not installed. You can install it by typing: sudo apt-get install python-software-properties andreas@j3515:~$ add-apt-repository is not available on all Ubuntu systems by default. It is provided by software-properties-common in more recent releases, and is seeded on server now. See bug 439566. $ seeded-in-ubuntu -b software-properties-common software-properties-common is seeded in: edubuntu: dvd kubuntu-active: daily-live kubuntu: daily-live lubuntu: daily, daily-live, daily-preinstalled mythbuntu: daily-live ubuntu-gnome: daily-live ubuntu-server: daily ubuntu-touch: daily-preinstalled ubuntu: daily-live ubuntukylin: daily-live ubuntustudio: dvd xubuntu: daily-live Where, specifically, is it missing? The current solution is easy and is available on virtually all installations, without the need to manually install anything. Providing easy access to the source is essential for a open source distribution. All this is true, but you haven't said specifically why this proposal is a problem, or why. This proposal *will* make source available without having to manually install anything. And providing easy access to the source *is* essential, but this proposal doesn't take easy access away. Others have said why the current situation is a problem, and you haven't addressed that here at all, given that add-apt-repository *is* available by default now. If add-apt-repository is not available by default, do you object to it *being* made available by default to resolve this? -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On Tuesday, July 23, 2013 08:12:16 AM Robie Basak wrote: On Tue, Jul 23, 2013 at 03:02:02AM -0400, Scott Kitterman wrote: So those are a couple of examples of what I think is definitely not what we want. I'm open to discussion about alternate ways to preserve easy access to the source. How about: $ sudo apt-get source hello Reading package lists... Done Building dependency tree Reading state information... Done E: You must put some 'source' URIs in your sources.list E: Type add-apt-repository sources to do this automatically for you. $ sudo add-apt-repository sources deb-src lines have been added to your sources.list. Now type apt-get update, and then apt-get source ... will work. $ sudo apt-get update (...) $ sudo apt-get source hello (works) To do this, we'd need to patch apt to add the second error line, and implement sources to add-apt-repository. Assuming add-apt-repository was installed by default, it's close. I think something like this might be reasonable (imagine some policykit or whatever it is called now magic here): $ sudo apt-get source hello Reading package lists... Done Building dependency tree Reading state information... Done E: You must put some 'source' URIs in your sources.list Would you like 'source' URIs to be added? (y/N) Y deb-src lines have been added to your sources.list. ... Get:9 http://archive.ubuntu.com saucy/main Sources [1,001 kB] Get:10 http://archive.ubuntu.com saucy/restricted Sources [6,578 B] Get:11 http://archive.ubuntu.com saucy/universe Sources [6,071 kB] ... apt-get source lightdm-kde Reading package lists... Done Building dependency tree Reading state information... Done NOTICE: 'lightdm-kde' packaging is maintained in the 'Git' version control system at: git://git.debian.org/pkg-kde/kde-extras/lightdm-kde.git Need to get 1,386 kB of source archives. Get:1 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (dsc) [1,543 B] Get:2 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (tar) [1,379 kB] Get:3 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (diff) [5,088 B] Fetched 1,386 kB in 1s (807 kB/s) apt-get source lightdm-kde Reading package lists... Done Building dependency tree Reading state information... Done NOTICE: 'lightdm-kde' packaging is maintained in the 'Git' version control system at: git://git.debian.org/pkg-kde/kde-extras/lightdm-kde.git Need to get 1,386 kB of source archives. Get:1 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (dsc) [1,543 B] Get:2 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (tar) [1,379 kB] Get:3 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (diff) [5,088 B] Fetched 1,386 kB in 1s (807 kB/s) (and so on) In other words, it's, I think, possible to make it roughly as easy as it is now to get source without having the sources.list cluttered. For users of our releases, I doubt it saves much, but that would be a way to do it that both avoids whatever amount of bandwidth usage is involved until the user opts in to it, but preserves ready access to the source that I think is important. Scott K -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On Tuesday, July 23, 2013 08:21:40 AM Jordon Bedwell wrote: On Tue, Jul 23, 2013 at 6:32 AM, Scott Kitterman ubu...@kitterman.com wrote: Assuming add-apt-repository was installed by default, it's close. I think something like this might be reasonable (imagine some policykit or whatever it is called now magic here): $ sudo apt-get source hello Reading package lists... Done Building dependency tree Reading state information... Done E: You must put some 'source' URIs in your sources.list Would you like 'source' URIs to be added? (y/N) Y deb-src lines have been added to your sources.list. ... Get:9 http://archive.ubuntu.com saucy/main Sources [1,001 kB] Get:10 http://archive.ubuntu.com saucy/restricted Sources [6,578 B] Get:11 http://archive.ubuntu.com saucy/universe Sources [6,071 kB] In other words, it's, I think, possible to make it roughly as easy as it is now to get source without having the sources.list cluttered. For users of our releases, I doubt it saves much, but that would be a way to do it that both avoids whatever amount of bandwidth usage is involved until the user opts in to it, but preserves ready access to the source that I think is important. Depending on how clever and one-off you want to be you could also just give them the http url to the source as well. It shouldn't be that hard to guess since apt already has most of the information needed to just generate the URL from a chosen apt server in the normal deb. This would allow for one-off downloads (for example somebody needs to look at the way debian does some of it's compiles so they can replicate without a package so they grab the source for nginx -- that's a one-off IMO if they would never use any other source package.) Though I personally like a default command that would be something like add-apt-default-sources so you can also give them the ability to run that command and disable sources too (but you can already do that via the GUI and terminal by editing /etc/apt/sources.list and such.) Before we run off and expend a lot more effort on this, I'd like to see something other than handwaving that this is really is a significant issue. /ubuntu/dists/raring-security/main/source [ ] Release 24-Jul-2013 01:16 106 [ ] Sources.bz2 24-Jul-2013 01:16 32K [ ] Sources.gz 24-Jul-2013 01:16 38K For end users, how much is really downloaded? /ubuntu/dists/raring-updates/main/source [ ] Release 24-Jul-2013 01:16 105 [ ] Sources.bz2 24-Jul-2013 01:16 50K [ ] Sources.gz 24-Jul-2013 01:16 62K /ubuntu/dists/raring-updates/universe/source [ ] Release 24-Jul-2013 01:16 109 [ ] Sources.bz2 24-Jul-2013 01:16 64K [ ] Sources.gz 24-Jul-2013 01:16 77K It doesn't seem like a lot. Scott K -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On 07/23/2013 12:02 AM, Scott Kitterman wrote: On Tuesday, July 23, 2013 06:59:43 AM Robie Basak wrote: On Tue, Jul 23, 2013 at 01:51:46AM -0400, Scott Kitterman wrote: I think most developers would believe the current situation is appropriate. I disagree. By default users have the same access to source and binary packages and for a free software distribution, that is the ethically correct approach. Indeed, but you never replied to my original response to your concern. By same access, do you specifically require the mechanism to be to keep users' local apt caches maintained with source entries? If so, why is such a mechanism necessary to fit the spirit of Free Software? If the user still has easy access to the source, why is this not sufficient? I'm happy to discuss what easy access might actually mean, but I see no reason that it should require the waste of users' bandwidth and time. Sorry. I didn't mean to ignore you. What's easy? For example, I think install more packages to get the tools to get the source (use pull-lp-source in ubuntu-dev-tools) doesn't qualify. There are tons of documentation all over the web and other places as well that assume apt-get source works. I agree, it would be nice to keep existing things working. So those are a couple of examples of what I think is definitely not what we want. I'm open to discussion about alternate ways to preserve easy access to the source. What if we disabled default source fetching but changed apt-get source to offer to turn them back on when it was run? -Scott Ritchie -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On Tuesday, July 23, 2013 09:19:36 PM Scott Ritchie wrote: On 07/23/2013 12:02 AM, Scott Kitterman wrote: On Tuesday, July 23, 2013 06:59:43 AM Robie Basak wrote: On Tue, Jul 23, 2013 at 01:51:46AM -0400, Scott Kitterman wrote: I think most developers would believe the current situation is appropriate. I disagree. By default users have the same access to source and binary packages and for a free software distribution, that is the ethically correct approach. Indeed, but you never replied to my original response to your concern. By same access, do you specifically require the mechanism to be to keep users' local apt caches maintained with source entries? If so, why is such a mechanism necessary to fit the spirit of Free Software? If the user still has easy access to the source, why is this not sufficient? I'm happy to discuss what easy access might actually mean, but I see no reason that it should require the waste of users' bandwidth and time. Sorry. I didn't mean to ignore you. What's easy? For example, I think install more packages to get the tools to get the source (use pull-lp-source in ubuntu-dev-tools) doesn't qualify. There are tons of documentation all over the web and other places as well that assume apt-get source works. I agree, it would be nice to keep existing things working. So those are a couple of examples of what I think is definitely not what we want. I'm open to discussion about alternate ways to preserve easy access to the source. What if we disabled default source fetching but changed apt-get source to offer to turn them back on when it was run? One other aspect of this that has occurred to me is that adding new sources (whehter they are present, but disabled or added new) takes administrator rights. Currently, any user of the system can get source using standard distro tools (apt-get). If you have to add a repository, you either have to be an admin or get one. Scott K -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On Tue, Jul 23, 2013 at 09:31:15PM -0400, Scott Kitterman wrote: Before we run off and expend a lot more effort on this, I'd like to see something other than handwaving that this is really is a significant issue. [size comparisions snipped] My concern is latency, not size. How many round trips will we save this way? For cloud images using Amazon S3 mirrors, for example, each request is quite a bit slower AIUI, and apt-get doesn't currently support concurrent requests to a single server. This is a pain for instances that start up with cloud-init and immediately have to update sources and install things before they can become functional. It'd be nice to see the delay from juju deploy to having a live service running get shorter. Same for juju add-unit. Admittedly an alternative means to achieve this could be to have cloud-init remove the deb-src lines first, but it seems a shame to leave others behind if this really does improve things. I agree that I should come up with actual figures before pushing ahead for this reason. -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On Wednesday, July 24, 2013 03:46:10 AM Robie Basak wrote: On Tue, Jul 23, 2013 at 09:31:15PM -0400, Scott Kitterman wrote: Before we run off and expend a lot more effort on this, I'd like to see something other than handwaving that this is really is a significant issue. [size comparisions snipped] My concern is latency, not size. How many round trips will we save this way? For cloud images using Amazon S3 mirrors, for example, each request is quite a bit slower AIUI, and apt-get doesn't currently support concurrent requests to a single server. This is a pain for instances that start up with cloud-init and immediately have to update sources and install things before they can become functional. It'd be nice to see the delay from juju deploy to having a live service running get shorter. Same for juju add-unit. Admittedly an alternative means to achieve this could be to have cloud-init remove the deb-src lines first, but it seems a shame to leave others behind if this really does improve things. I agree that I should come up with actual figures before pushing ahead for this reason. Has any work been done on concurrent requests? That would likely be pretty broadly useful, not just for cloud images. Scott K -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
Or 90/110K per day per computer for Precise. I guess what was getting me is the additional 6-7MB during install or first update: http://archive.ubuntu.com/ubuntu/dists/precise/universe/source/ 4.8M/5.9M http://archive.ubuntu.com/ubuntu/dists/precise/main/source/ 912K/1.1M On 24 July 2013 09:31, Scott Kitterman ubu...@kitterman.com wrote: On Tuesday, July 23, 2013 08:21:40 AM Jordon Bedwell wrote: On Tue, Jul 23, 2013 at 6:32 AM, Scott Kitterman ubu...@kitterman.com wrote: Assuming add-apt-repository was installed by default, it's close. I think something like this might be reasonable (imagine some policykit or whatever it is called now magic here): $ sudo apt-get source hello Reading package lists... Done Building dependency tree Reading state information... Done E: You must put some 'source' URIs in your sources.list Would you like 'source' URIs to be added? (y/N) Y deb-src lines have been added to your sources.list. ... Get:9 http://archive.ubuntu.com saucy/main Sources [1,001 kB] Get:10 http://archive.ubuntu.com saucy/restricted Sources [6,578 B] Get:11 http://archive.ubuntu.com saucy/universe Sources [6,071 kB] In other words, it's, I think, possible to make it roughly as easy as it is now to get source without having the sources.list cluttered. For users of our releases, I doubt it saves much, but that would be a way to do it that both avoids whatever amount of bandwidth usage is involved until the user opts in to it, but preserves ready access to the source that I think is important. Depending on how clever and one-off you want to be you could also just give them the http url to the source as well. It shouldn't be that hard to guess since apt already has most of the information needed to just generate the URL from a chosen apt server in the normal deb. This would allow for one-off downloads (for example somebody needs to look at the way debian does some of it's compiles so they can replicate without a package so they grab the source for nginx -- that's a one-off IMO if they would never use any other source package.) Though I personally like a default command that would be something like add-apt-default-sources so you can also give them the ability to run that command and disable sources too (but you can already do that via the GUI and terminal by editing /etc/apt/sources.list and such.) Before we run off and expend a lot more effort on this, I'd like to see something other than handwaving that this is really is a significant issue. /ubuntu/dists/raring-security/main/source [ ] Release 24-Jul-2013 01:16 106 [ ] Sources.bz2 24-Jul-2013 01:16 32K [ ] Sources.gz 24-Jul-2013 01:16 38K For end users, how much is really downloaded? /ubuntu/dists/raring-updates/main/source [ ] Release 24-Jul-2013 01:16 105 [ ] Sources.bz2 24-Jul-2013 01:16 50K [ ] Sources.gz 24-Jul-2013 01:16 62K /ubuntu/dists/raring-updates/universe/source [ ] Release 24-Jul-2013 01:16 109 [ ] Sources.bz2 24-Jul-2013 01:16 64K [ ] Sources.gz 24-Jul-2013 01:16 77K It doesn't seem like a lot. Scott K -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel -- Daniel J Blueman -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
Perhaps we have two issues here: - the download during installs or first index update is 6-7MB extra, which makes a real difference when installing lots of computers - downloads from security.ubuntu.com being slow (eg 1-5KB/s) as it's 500ms away The 20% additional download due to sources [1] would help both issues, but perhaps of bigger impact, trusting the country-level mirror for the security updates? Daniel --- [1] Get:1 http://security.ubuntu.com precise-security Release.gpg [198 B] Get:2 http://security.ubuntu.com precise-security Release [49.6 kB] Get:3 http://security.ubuntu.com precise-security/main Sources [83.5 kB] Get:4 http://security.ubuntu.com precise-security/restricted Sources [2494 B] Get:5 http://security.ubuntu.com precise-security/universe Sources [27.1 kB] Get:6 http://security.ubuntu.com precise-security/multiverse Sources [1383 B] Get:7 http://security.ubuntu.com precise-security/main amd64 Packages [296 kB] Get:8 http://security.ubuntu.com precise-security/restricted amd64 Packages [4627 B] Get:9 http://security.ubuntu.com precise-security/universe amd64 Packages [77.7 kB] Get:10 http://security.ubuntu.com precise-security/multiverse amd64 Packages [2186 B] Get:11 http://security.ubuntu.com precise-security/main i386 Packages [311 kB] Get:12 http://security.ubuntu.com precise-security/restricted i386 Packages [4620 B] Get:13 http://security.ubuntu.com precise-security/universe i386 Packages [80.5 kB] Get:14 http://security.ubuntu.com precise-security/multiverse i386 Packages [2371 B] Get:15 http://security.ubuntu.com precise-security/main TranslationIndex [74 B] Get:16 http://security.ubuntu.com precise-security/multiverse TranslationIndex [71 B] Get:17 http://security.ubuntu.com precise-security/restricted TranslationIndex [72 B] Get:18 http://security.ubuntu.com precise-security/universe TranslationIndex [73 B] On 24 July 2013 10:46, Robie Basak robie.ba...@ubuntu.com wrote: On Tue, Jul 23, 2013 at 09:31:15PM -0400, Scott Kitterman wrote: Before we run off and expend a lot more effort on this, I'd like to see something other than handwaving that this is really is a significant issue. [size comparisions snipped] My concern is latency, not size. How many round trips will we save this way? For cloud images using Amazon S3 mirrors, for example, each request is quite a bit slower AIUI, and apt-get doesn't currently support concurrent requests to a single server. This is a pain for instances that start up with cloud-init and immediately have to update sources and install things before they can become functional. It'd be nice to see the delay from juju deploy to having a live service running get shorter. Same for juju add-unit. Admittedly an alternative means to achieve this could be to have cloud-init remove the deb-src lines first, but it seems a shame to leave others behind if this really does improve things. I agree that I should come up with actual figures before pushing ahead for this reason. -- Ubuntu-devel-discuss mailing list ubuntu-devel-disc...@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss -- Daniel J Blueman -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On 24 July 2013 11:08, Scott Kitterman ubu...@kitterman.com wrote: On Wednesday, July 24, 2013 11:00:40 AM Daniel J Blueman wrote: Perhaps we have two issues here: The 20% additional download due to sources [1] would help both issues, but perhaps of bigger impact, trusting the country-level mirror for the security updates? ... You aren't. Security updates are pushed first to security.ubuntu.com and then copied to archive.ubuntu.com and mirrored from there. The security pocket isn't mirrored so you always hit it directly and if a country mirror lags, you get the package from security.ubuntu.com. Also, the signing key is the same Ubuntu archive signing key whether you're getting a package form archive.ubuntu.com or a country mirror, so you aren't trusting the country mirror cryptographically either. What I meant, if the country-level archive is sync'd every 12-24 hours, would it be sufficient to download the security pocket from cc.archive.ubuntu.com? It is mirrored, so this would alleviate the second issue. Daniel -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
On Tue, Jul 23, 2013 at 01:51:46AM -0400, Scott Kitterman wrote: I think most developers would believe the current situation is appropriate. I disagree. By default users have the same access to source and binary packages and for a free software distribution, that is the ethically correct approach. Indeed, but you never replied to my original response to your concern. By same access, do you specifically require the mechanism to be to keep users' local apt caches maintained with source entries? If so, why is such a mechanism necessary to fit the spirit of Free Software? If the user still has easy access to the source, why is this not sufficient? I'm happy to discuss what easy access might actually mean, but I see no reason that it should require the waste of users' bandwidth and time. Robie -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
(pardon the top-posting) I think the slight reduction in ethics (relevant mainly to developers) is a good trade to help deployability in the real world. We'll leave sources enabled by default for development releases. For the other 99% of users, where practicality is more important than immediate access to source, we end up wasting ~10% of Canonical and our mirror's bandwidth on the source updates. This makes a difference when behind a congested network, running on battery or so on. That 10% when accessing security.ubuntu.com really helps, particularly when topologically distant from the UK (if you have good network connectivity, ask someone who hasn't got it). No? On 23 July 2013 13:51, Scott Kitterman ubu...@kitterman.com wrote: On Tuesday, July 23, 2013 11:02:00 AM Daniel J Blueman wrote: By large, developers are uninterested in this, but it is important for users and where we use Ubuntu. Anyone care to comment on how we can progress this? I think most developers would believe the current situation is appropriate. By default users have the same access to source and binary packages and for a free software distribution, that is the ethically correct approach. Scott K On 15 July 2013 13:32, Daniel J Blueman dan...@quora.org wrote: From earlier feedback, there were no overriding reasons why package sources should be enabled by default. We not only save congestion on security.ubuntu.com, but quite a lot of country-level mirrors point to Canonical's servers, which are relatively distant and slow (~80KB/s from here), so this is a win. So, what's the path to change this? On 21 May 2013 22:04, J Fernyhough j.fernyho...@gmail.com wrote: On 21 May 2013 13:55, Robie Basak robie.ba...@canonical.com wrote: What if we provided a reasonable message if no deb-src lines are defined, with a single simple command to add them and run apt-get update for you? I don't think it would even need that - software-properties (Software Updates) already has the necessary checkbox. All that is needed to enable sources is to tick that box. From a technical point of view, does mirroring the deb lines into deb-src lines work in all cases? Would doing so break anything? This is effectively what Software Sources does under-the-hood. I have to agree, if the amount being downloaded is not trivial (which I thought it was) then there's no need to have them enabled by default when it's very easy to turn them on. One of the first things I do on any new install is disable those that aren't needed. Jonathon (to the list this time) -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss -- Daniel J Blueman -- ubuntu-devel mailing list ubuntu-de...@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel -- Daniel J Blueman -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Tuesday, July 23, 2013 06:59:43 AM Robie Basak wrote: On Tue, Jul 23, 2013 at 01:51:46AM -0400, Scott Kitterman wrote: I think most developers would believe the current situation is appropriate. I disagree. By default users have the same access to source and binary packages and for a free software distribution, that is the ethically correct approach. Indeed, but you never replied to my original response to your concern. By same access, do you specifically require the mechanism to be to keep users' local apt caches maintained with source entries? If so, why is such a mechanism necessary to fit the spirit of Free Software? If the user still has easy access to the source, why is this not sufficient? I'm happy to discuss what easy access might actually mean, but I see no reason that it should require the waste of users' bandwidth and time. Sorry. I didn't mean to ignore you. What's easy? For example, I think install more packages to get the tools to get the source (use pull-lp-source in ubuntu-dev-tools) doesn't qualify. There are tons of documentation all over the web and other places as well that assume apt-get source works. I think access using installed tools that are normally used for the job (wget is installed (I think) by default, but I don't think having to go to a web page to find a URL and then wget'ing the components of the source package is easy either. So those are a couple of examples of what I think is definitely not what we want. I'm open to discussion about alternate ways to preserve easy access to the source. Scott K -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Tue, Jul 23, 2013 at 03:02:02AM -0400, Scott Kitterman wrote: So those are a couple of examples of what I think is definitely not what we want. I'm open to discussion about alternate ways to preserve easy access to the source. How about: $ sudo apt-get source hello Reading package lists... Done Building dependency tree Reading state information... Done E: You must put some 'source' URIs in your sources.list E: Type add-apt-repository sources to do this automatically for you. $ sudo add-apt-repository sources deb-src lines have been added to your sources.list. Now type apt-get update, and then apt-get source ... will work. $ sudo apt-get update (...) $ sudo apt-get source hello (works) To do this, we'd need to patch apt to add the second error line, and implement sources to add-apt-repository. Robie -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Hi Daniel (2013.07.23_08:13:47_+0200) For the other 99% of users, where practicality is more important than immediate access to source, we end up wasting ~10% of Canonical and our mirror's bandwidth on the source updates. Can you back that up with evidence? As I (and a few other people) have repeatedly said in this thread: The release pocket lists aren't changed after release. Only -updates, -security, -backports and -proposed change, and they are all small because they are an overlay on the release pocket. SR -- Stefano Rivera http://tumbleweed.org.za/ H: +27 21 461 1230 C: +27 72 419 8559 -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Tuesday, July 23, 2013 08:12:16 AM Robie Basak wrote: On Tue, Jul 23, 2013 at 03:02:02AM -0400, Scott Kitterman wrote: So those are a couple of examples of what I think is definitely not what we want. I'm open to discussion about alternate ways to preserve easy access to the source. How about: $ sudo apt-get source hello Reading package lists... Done Building dependency tree Reading state information... Done E: You must put some 'source' URIs in your sources.list E: Type add-apt-repository sources to do this automatically for you. $ sudo add-apt-repository sources deb-src lines have been added to your sources.list. Now type apt-get update, and then apt-get source ... will work. $ sudo apt-get update (...) $ sudo apt-get source hello (works) To do this, we'd need to patch apt to add the second error line, and implement sources to add-apt-repository. Assuming add-apt-repository was installed by default, it's close. I think something like this might be reasonable (imagine some policykit or whatever it is called now magic here): $ sudo apt-get source hello Reading package lists... Done Building dependency tree Reading state information... Done E: You must put some 'source' URIs in your sources.list Would you like 'source' URIs to be added? (y/N) Y deb-src lines have been added to your sources.list. ... Get:9 http://archive.ubuntu.com saucy/main Sources [1,001 kB] Get:10 http://archive.ubuntu.com saucy/restricted Sources [6,578 B] Get:11 http://archive.ubuntu.com saucy/universe Sources [6,071 kB] ... apt-get source lightdm-kde Reading package lists... Done Building dependency tree Reading state information... Done NOTICE: 'lightdm-kde' packaging is maintained in the 'Git' version control system at: git://git.debian.org/pkg-kde/kde-extras/lightdm-kde.git Need to get 1,386 kB of source archives. Get:1 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (dsc) [1,543 B] Get:2 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (tar) [1,379 kB] Get:3 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (diff) [5,088 B] Fetched 1,386 kB in 1s (807 kB/s) apt-get source lightdm-kde Reading package lists... Done Building dependency tree Reading state information... Done NOTICE: 'lightdm-kde' packaging is maintained in the 'Git' version control system at: git://git.debian.org/pkg-kde/kde-extras/lightdm-kde.git Need to get 1,386 kB of source archives. Get:1 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (dsc) [1,543 B] Get:2 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (tar) [1,379 kB] Get:3 http://archive.ubuntu.com/ubuntu/ saucy/universe lightdm-kde 0.3.2.1-1ubuntu2 (diff) [5,088 B] Fetched 1,386 kB in 1s (807 kB/s) (and so on) In other words, it's, I think, possible to make it roughly as easy as it is now to get source without having the sources.list cluttered. For users of our releases, I doubt it saves much, but that would be a way to do it that both avoids whatever amount of bandwidth usage is involved until the user opts in to it, but preserves ready access to the source that I think is important. Scott K -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Tuesday, July 23, 2013 08:21:40 AM Jordon Bedwell wrote: On Tue, Jul 23, 2013 at 6:32 AM, Scott Kitterman ubu...@kitterman.com wrote: Assuming add-apt-repository was installed by default, it's close. I think something like this might be reasonable (imagine some policykit or whatever it is called now magic here): $ sudo apt-get source hello Reading package lists... Done Building dependency tree Reading state information... Done E: You must put some 'source' URIs in your sources.list Would you like 'source' URIs to be added? (y/N) Y deb-src lines have been added to your sources.list. ... Get:9 http://archive.ubuntu.com saucy/main Sources [1,001 kB] Get:10 http://archive.ubuntu.com saucy/restricted Sources [6,578 B] Get:11 http://archive.ubuntu.com saucy/universe Sources [6,071 kB] In other words, it's, I think, possible to make it roughly as easy as it is now to get source without having the sources.list cluttered. For users of our releases, I doubt it saves much, but that would be a way to do it that both avoids whatever amount of bandwidth usage is involved until the user opts in to it, but preserves ready access to the source that I think is important. Depending on how clever and one-off you want to be you could also just give them the http url to the source as well. It shouldn't be that hard to guess since apt already has most of the information needed to just generate the URL from a chosen apt server in the normal deb. This would allow for one-off downloads (for example somebody needs to look at the way debian does some of it's compiles so they can replicate without a package so they grab the source for nginx -- that's a one-off IMO if they would never use any other source package.) Though I personally like a default command that would be something like add-apt-default-sources so you can also give them the ability to run that command and disable sources too (but you can already do that via the GUI and terminal by editing /etc/apt/sources.list and such.) Before we run off and expend a lot more effort on this, I'd like to see something other than handwaving that this is really is a significant issue. /ubuntu/dists/raring-security/main/source [ ] Release 24-Jul-2013 01:16 106 [ ] Sources.bz2 24-Jul-2013 01:16 32K [ ] Sources.gz 24-Jul-2013 01:16 38K For end users, how much is really downloaded? /ubuntu/dists/raring-updates/main/source [ ] Release 24-Jul-2013 01:16 105 [ ] Sources.bz2 24-Jul-2013 01:16 50K [ ] Sources.gz 24-Jul-2013 01:16 62K /ubuntu/dists/raring-updates/universe/source [ ] Release 24-Jul-2013 01:16 109 [ ] Sources.bz2 24-Jul-2013 01:16 64K [ ] Sources.gz 24-Jul-2013 01:16 77K It doesn't seem like a lot. Scott K -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Or 90/110K per day per computer for Precise. I guess what was getting me is the additional 6-7MB during install or first update: http://archive.ubuntu.com/ubuntu/dists/precise/universe/source/ 4.8M/5.9M http://archive.ubuntu.com/ubuntu/dists/precise/main/source/ 912K/1.1M On 24 July 2013 09:31, Scott Kitterman ubu...@kitterman.com wrote: On Tuesday, July 23, 2013 08:21:40 AM Jordon Bedwell wrote: On Tue, Jul 23, 2013 at 6:32 AM, Scott Kitterman ubu...@kitterman.com wrote: Assuming add-apt-repository was installed by default, it's close. I think something like this might be reasonable (imagine some policykit or whatever it is called now magic here): $ sudo apt-get source hello Reading package lists... Done Building dependency tree Reading state information... Done E: You must put some 'source' URIs in your sources.list Would you like 'source' URIs to be added? (y/N) Y deb-src lines have been added to your sources.list. ... Get:9 http://archive.ubuntu.com saucy/main Sources [1,001 kB] Get:10 http://archive.ubuntu.com saucy/restricted Sources [6,578 B] Get:11 http://archive.ubuntu.com saucy/universe Sources [6,071 kB] In other words, it's, I think, possible to make it roughly as easy as it is now to get source without having the sources.list cluttered. For users of our releases, I doubt it saves much, but that would be a way to do it that both avoids whatever amount of bandwidth usage is involved until the user opts in to it, but preserves ready access to the source that I think is important. Depending on how clever and one-off you want to be you could also just give them the http url to the source as well. It shouldn't be that hard to guess since apt already has most of the information needed to just generate the URL from a chosen apt server in the normal deb. This would allow for one-off downloads (for example somebody needs to look at the way debian does some of it's compiles so they can replicate without a package so they grab the source for nginx -- that's a one-off IMO if they would never use any other source package.) Though I personally like a default command that would be something like add-apt-default-sources so you can also give them the ability to run that command and disable sources too (but you can already do that via the GUI and terminal by editing /etc/apt/sources.list and such.) Before we run off and expend a lot more effort on this, I'd like to see something other than handwaving that this is really is a significant issue. /ubuntu/dists/raring-security/main/source [ ] Release 24-Jul-2013 01:16 106 [ ] Sources.bz2 24-Jul-2013 01:16 32K [ ] Sources.gz 24-Jul-2013 01:16 38K For end users, how much is really downloaded? /ubuntu/dists/raring-updates/main/source [ ] Release 24-Jul-2013 01:16 105 [ ] Sources.bz2 24-Jul-2013 01:16 50K [ ] Sources.gz 24-Jul-2013 01:16 62K /ubuntu/dists/raring-updates/universe/source [ ] Release 24-Jul-2013 01:16 109 [ ] Sources.bz2 24-Jul-2013 01:16 64K [ ] Sources.gz 24-Jul-2013 01:16 77K It doesn't seem like a lot. Scott K -- ubuntu-devel mailing list ubuntu-de...@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel -- Daniel J Blueman -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Tue, Jul 23, 2013 at 09:31:15PM -0400, Scott Kitterman wrote: Before we run off and expend a lot more effort on this, I'd like to see something other than handwaving that this is really is a significant issue. [size comparisions snipped] My concern is latency, not size. How many round trips will we save this way? For cloud images using Amazon S3 mirrors, for example, each request is quite a bit slower AIUI, and apt-get doesn't currently support concurrent requests to a single server. This is a pain for instances that start up with cloud-init and immediately have to update sources and install things before they can become functional. It'd be nice to see the delay from juju deploy to having a live service running get shorter. Same for juju add-unit. Admittedly an alternative means to achieve this could be to have cloud-init remove the deb-src lines first, but it seems a shame to leave others behind if this really does improve things. I agree that I should come up with actual figures before pushing ahead for this reason. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Perhaps we have two issues here: - the download during installs or first index update is 6-7MB extra, which makes a real difference when installing lots of computers - downloads from security.ubuntu.com being slow (eg 1-5KB/s) as it's 500ms away The 20% additional download due to sources [1] would help both issues, but perhaps of bigger impact, trusting the country-level mirror for the security updates? Daniel --- [1] Get:1 http://security.ubuntu.com precise-security Release.gpg [198 B] Get:2 http://security.ubuntu.com precise-security Release [49.6 kB] Get:3 http://security.ubuntu.com precise-security/main Sources [83.5 kB] Get:4 http://security.ubuntu.com precise-security/restricted Sources [2494 B] Get:5 http://security.ubuntu.com precise-security/universe Sources [27.1 kB] Get:6 http://security.ubuntu.com precise-security/multiverse Sources [1383 B] Get:7 http://security.ubuntu.com precise-security/main amd64 Packages [296 kB] Get:8 http://security.ubuntu.com precise-security/restricted amd64 Packages [4627 B] Get:9 http://security.ubuntu.com precise-security/universe amd64 Packages [77.7 kB] Get:10 http://security.ubuntu.com precise-security/multiverse amd64 Packages [2186 B] Get:11 http://security.ubuntu.com precise-security/main i386 Packages [311 kB] Get:12 http://security.ubuntu.com precise-security/restricted i386 Packages [4620 B] Get:13 http://security.ubuntu.com precise-security/universe i386 Packages [80.5 kB] Get:14 http://security.ubuntu.com precise-security/multiverse i386 Packages [2371 B] Get:15 http://security.ubuntu.com precise-security/main TranslationIndex [74 B] Get:16 http://security.ubuntu.com precise-security/multiverse TranslationIndex [71 B] Get:17 http://security.ubuntu.com precise-security/restricted TranslationIndex [72 B] Get:18 http://security.ubuntu.com precise-security/universe TranslationIndex [73 B] On 24 July 2013 10:46, Robie Basak robie.ba...@ubuntu.com wrote: On Tue, Jul 23, 2013 at 09:31:15PM -0400, Scott Kitterman wrote: Before we run off and expend a lot more effort on this, I'd like to see something other than handwaving that this is really is a significant issue. [size comparisions snipped] My concern is latency, not size. How many round trips will we save this way? For cloud images using Amazon S3 mirrors, for example, each request is quite a bit slower AIUI, and apt-get doesn't currently support concurrent requests to a single server. This is a pain for instances that start up with cloud-init and immediately have to update sources and install things before they can become functional. It'd be nice to see the delay from juju deploy to having a live service running get shorter. Same for juju add-unit. Admittedly an alternative means to achieve this could be to have cloud-init remove the deb-src lines first, but it seems a shame to leave others behind if this really does improve things. I agree that I should come up with actual figures before pushing ahead for this reason. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss -- Daniel J Blueman -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Wednesday, July 24, 2013 11:00:40 AM Daniel J Blueman wrote: Perhaps we have two issues here: The 20% additional download due to sources [1] would help both issues, but perhaps of bigger impact, trusting the country-level mirror for the security updates? ... You aren't. Security updates are pushed first to security.ubuntu.com and then copied to archive.ubuntu.com and mirrored from there. The security pocket isn't mirrored so you always hit it directly and if a country mirror lags, you get the package from security.ubuntu.com. Also, the signing key is the same Ubuntu archive signing key whether you're getting a package form archive.ubuntu.com or a country mirror, so you aren't trusting the country mirror cryptographically either. Scott K -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On 24 July 2013 11:08, Scott Kitterman ubu...@kitterman.com wrote: On Wednesday, July 24, 2013 11:00:40 AM Daniel J Blueman wrote: Perhaps we have two issues here: The 20% additional download due to sources [1] would help both issues, but perhaps of bigger impact, trusting the country-level mirror for the security updates? ... You aren't. Security updates are pushed first to security.ubuntu.com and then copied to archive.ubuntu.com and mirrored from there. The security pocket isn't mirrored so you always hit it directly and if a country mirror lags, you get the package from security.ubuntu.com. Also, the signing key is the same Ubuntu archive signing key whether you're getting a package form archive.ubuntu.com or a country mirror, so you aren't trusting the country mirror cryptographically either. What I meant, if the country-level archive is sync'd every 12-24 hours, would it be sufficient to download the security pocket from cc.archive.ubuntu.com? It is mirrored, so this would alleviate the second issue. Daniel -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
By large, developers are uninterested in this, but it is important for users and where we use Ubuntu. Anyone care to comment on how we can progress this? On 15 July 2013 13:32, Daniel J Blueman dan...@quora.org wrote: From earlier feedback, there were no overriding reasons why package sources should be enabled by default. We not only save congestion on security.ubuntu.com, but quite a lot of country-level mirrors point to Canonical's servers, which are relatively distant and slow (~80KB/s from here), so this is a win. So, what's the path to change this? On 21 May 2013 22:04, J Fernyhough j.fernyho...@gmail.com wrote: On 21 May 2013 13:55, Robie Basak robie.ba...@canonical.com wrote: What if we provided a reasonable message if no deb-src lines are defined, with a single simple command to add them and run apt-get update for you? I don't think it would even need that - software-properties (Software Updates) already has the necessary checkbox. All that is needed to enable sources is to tick that box. From a technical point of view, does mirroring the deb lines into deb-src lines work in all cases? Would doing so break anything? This is effectively what Software Sources does under-the-hood. I have to agree, if the amount being downloaded is not trivial (which I thought it was) then there's no need to have them enabled by default when it's very easy to turn them on. One of the first things I do on any new install is disable those that aren't needed. Jonathon (to the list this time) -- Ubuntu-devel-discuss mailing list ubuntu-devel-disc...@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss -- Daniel J Blueman -- Daniel J Blueman -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Source packages appropriate by default?
By large, developers are uninterested in this, but it is important for users and where we use Ubuntu. Anyone care to comment on how we can progress this? On 15 July 2013 13:32, Daniel J Blueman dan...@quora.org wrote: From earlier feedback, there were no overriding reasons why package sources should be enabled by default. We not only save congestion on security.ubuntu.com, but quite a lot of country-level mirrors point to Canonical's servers, which are relatively distant and slow (~80KB/s from here), so this is a win. So, what's the path to change this? On 21 May 2013 22:04, J Fernyhough j.fernyho...@gmail.com wrote: On 21 May 2013 13:55, Robie Basak robie.ba...@canonical.com wrote: What if we provided a reasonable message if no deb-src lines are defined, with a single simple command to add them and run apt-get update for you? I don't think it would even need that - software-properties (Software Updates) already has the necessary checkbox. All that is needed to enable sources is to tick that box. From a technical point of view, does mirroring the deb lines into deb-src lines work in all cases? Would doing so break anything? This is effectively what Software Sources does under-the-hood. I have to agree, if the amount being downloaded is not trivial (which I thought it was) then there's no need to have them enabled by default when it's very easy to turn them on. One of the first things I do on any new install is disable those that aren't needed. Jonathon (to the list this time) -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss -- Daniel J Blueman -- Daniel J Blueman -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Tuesday, July 23, 2013 11:02:00 AM Daniel J Blueman wrote: By large, developers are uninterested in this, but it is important for users and where we use Ubuntu. Anyone care to comment on how we can progress this? I think most developers would believe the current situation is appropriate. By default users have the same access to source and binary packages and for a free software distribution, that is the ethically correct approach. Scott K On 15 July 2013 13:32, Daniel J Blueman dan...@quora.org wrote: From earlier feedback, there were no overriding reasons why package sources should be enabled by default. We not only save congestion on security.ubuntu.com, but quite a lot of country-level mirrors point to Canonical's servers, which are relatively distant and slow (~80KB/s from here), so this is a win. So, what's the path to change this? On 21 May 2013 22:04, J Fernyhough j.fernyho...@gmail.com wrote: On 21 May 2013 13:55, Robie Basak robie.ba...@canonical.com wrote: What if we provided a reasonable message if no deb-src lines are defined, with a single simple command to add them and run apt-get update for you? I don't think it would even need that - software-properties (Software Updates) already has the necessary checkbox. All that is needed to enable sources is to tick that box. From a technical point of view, does mirroring the deb lines into deb-src lines work in all cases? Would doing so break anything? This is effectively what Software Sources does under-the-hood. I have to agree, if the amount being downloaded is not trivial (which I thought it was) then there's no need to have them enabled by default when it's very easy to turn them on. One of the first things I do on any new install is disable those that aren't needed. Jonathon (to the list this time) -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss -- Daniel J Blueman -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
From earlier feedback, there were no overriding reasons why package sources should be enabled by default. We not only save congestion on security.ubuntu.com, but quite a lot of country-level mirrors point to Canonical's servers, which are relatively distant and slow (~80KB/s from here), so this is a win. So, what's the path to change this? On 21 May 2013 22:04, J Fernyhough j.fernyho...@gmail.com wrote: On 21 May 2013 13:55, Robie Basak robie.ba...@canonical.com wrote: What if we provided a reasonable message if no deb-src lines are defined, with a single simple command to add them and run apt-get update for you? I don't think it would even need that - software-properties (Software Updates) already has the necessary checkbox. All that is needed to enable sources is to tick that box. From a technical point of view, does mirroring the deb lines into deb-src lines work in all cases? Would doing so break anything? This is effectively what Software Sources does under-the-hood. I have to agree, if the amount being downloaded is not trivial (which I thought it was) then there's no need to have them enabled by default when it's very easy to turn them on. One of the first things I do on any new install is disable those that aren't needed. Jonathon (to the list this time) -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss -- Daniel J Blueman -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
unsubscribe On Mon, May 20, 2013 at 11:09 PM, Daniel J Blueman dan...@quora.org wrote: When installing Ubuntu, I always see the source packages enabled by default. For all the general users I install Ubuntu for (including servers), it's an utter waste of bandwidth for everyone, particularly when automatically checking once a day. This is amplified eg in schools without transparent webcaches etc. Anyone get the same feeling that we should have source packages an opt-in? Daniel -- Daniel J Blueman -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Wed, May 22, 2013 at 02:20:51AM +0200, Florian Diesch wrote: Am Mon, 20 May 2013 10:02:41 -0700 schrieb Benjamin Kerensa bkere...@ubuntu.com: I think in most parts of the world 4MB is trivial overhead for a user. Over here in German cheap mobile data tarrifs often get you something like a few hundered MByte/month. In some rural areas it's hard to get better than 64k/s if you don't want to setup your own radio link. 4MB every few days could quite hurt you with that. My flat outside Belfast has 20 meg of bandwidth. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Wed, 22 May 2013, Dale Amon wrote: On Wed, May 22, 2013 at 02:20:51AM +0200, Florian Diesch wrote: Am Mon, 20 May 2013 10:02:41 -0700 4MB every few days could quite hurt you with that. My flat outside Belfast has 20 meg of bandwidth. It is good that a decade[1] of broadband investment[2] has enabled good speeds, at least around Belfast. Access to decent speeds should enable opportunities that would not have been there before. Not all corners of the globe have received such levels of external investment... my boat uses a SIM card as primary downlink. At 03:00 in the morning I can usually get 8Mb/s, at the driving peaks of 08:00 and 18:00 I often get under 1Kb/s, with excruciating package loss, and multi-second latency. Unsurprisingly, I generally want to ignore the offers to update when I need to prioritise that little bandwidth for typing and IRC communication. I certainly don't want source code coming down. When I do need it, I normally need revision control data too, not just a tarball. In these circumstances, a git clone/bzr branch is a more efficient way to get that. -Paul [1] http://www.theregister.co.uk/2003/07/23/northern_ireland_aims_for_100pc/ [2] http://www.detini.gov.uk/deti-telecoms-index/deti-telecoms-broadband-fund.htm -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Another idea about mechanism: It seems to me that apt-get update does two logical tasks which serve two different use cases for many users, one of which is rare and often unnecessary. How about an apt configuration option that, when enabled (default: off), disables source index downloads when apt-get update is not used with an additional argument like apt-get --include-source update? Or just apt-get update-sources? A subsequent apt-get source could then warn you, in the error message that no sources are defined, that the configuration option to disable source index downloads is set, and the correct command to override it. Caveat 1: users who want to enable sources all the time would then need to edit (and know to edit) /etc/apt/apt.conf.d/10sources or something. Caveat 2: what other tools need to modified to honour this? Is it too onerous to patch them all? Just a thought. Removing or manipulating deb-src lines does seem easier. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Am Mittwoch, den 22.05.2013, 10:11 +0100 schrieb Robie Basak: Another idea about mechanism: It seems to me that apt-get update does two logical tasks which serve two different use cases for many users, one of which is rare and often unnecessary. How about an apt configuration option that, when enabled (default: off), disables source index downloads when apt-get update is not used with an additional argument like apt-get --include-source update? Or just apt-get update-sources? A subsequent apt-get source could then warn you, in the error message that no sources are defined, that the configuration option to disable source index downloads is set, and the correct command to override it. Caveat 1: users who want to enable sources all the time would then need to edit (and know to edit) /etc/apt/apt.conf.d/10sources or something. Caveat 2: what other tools need to modified to honour this? Is it too onerous to patch them all? Just a thought. Removing or manipulating deb-src lines does seem easier. Commenting/Uncommenting deb-src lines in /etc/apt/sources.list seems much simpler/easier. -- Benjamin Drung Debian Ubuntu Developer -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Wed, May 22, 2013 at 01:56:08PM +0200, Benjamin Drung wrote: Commenting/Uncommenting deb-src lines in /etc/apt/sources.list seems much simpler/easier. I can deal with that... I always have changes to make to sources.list anyway, so uncommenting a few more items is not an issue. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Wed, May 22, 2013 at 1:54 PM, Dale Amon a...@vnl.com wrote: On Wed, May 22, 2013 at 01:56:08PM +0200, Benjamin Drung wrote: Commenting/Uncommenting deb-src lines in /etc/apt/sources.list seems much simpler/easier. I can deal with that... I always have changes to make to sources.list anyway, so uncommenting a few more items is not an issue. To add to this commenting issue. This morning I reinstalled 13.04 and decided to see if there was really a difference when disabling the sources, talk about a massive speed up of an apt update. To me the reduction of time spent doing an apt update was so big I built a bash function to enable and disable them as I need them. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Benjamin Kerensa wrote on 20/05/13 18:02: On May 20, 2013 8:10 AM, Daniel J Blueman dan...@quora.org ... For all the general users I install Ubuntu for (including servers), it's an utter waste of bandwidth for everyone, particularly when automatically checking once a day. This is amplified eg in schools without transparent webcaches etc. Anyone get the same feeling that we should have source packages an opt-in? I think in most parts of the world 4MB is trivial overhead for a user. Although perhaps we should consider it since some developing nations have limited bandwidth? ... With every extra kilobyte required to check for updates, the check will complete measurably less often. Maybe the notebook lid will be closed, maybe the connection will time out, or maybe the computer will be shut down altogether. (Measurably, but not measuredly: we aren't collecting stats on this. See http://www.lukew.com/ff/entry.asp?1553 for examples of how even tiny changes in download times affect Web site success rates.) When update checks are completed less often, updates are installed later after they are issued. So, unnecessary downloads when checking for updates is a security issue. http://launchpad.net/bugs/74747 https://wiki.ubuntu.com/SoftwareUpdates#Ideas - -- mpt -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlGbQ/QACgkQ6PUxNfU6ecqVsgCdEyP61EWfH42SkFj4Z88Il3u9 MQcAnR5SU/6sBEE7PFuyxl707VH9oSPJ =nycT -END PGP SIGNATURE- -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
I propose we either disable source downloading by default at release time, but I conclude that developers generally don't care about this extra overhead (as we have a good setup). If really we can't see this from a user PoV, I'm happy to start a user discussion and see how users feel...? -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
(with my personal hat on) On Mon, May 20, 2013 at 01:48:47PM -0400, Scott Kitterman wrote: I think that's legally sufficient, but not in the spirit of free software. I agree that it's legally sufficient. I also agree that making the sources easily available for download is part of the essential spirit of free software. But I think that the minutiae of the exact mechanism used is not part of this spirit. There is an important distinction between policy and mechanism here. The spirit of free software clearly does not include mechanism. *Provided that it continues to be just as easy*, I don't think that making a change to this area breaks the spirit of free software. Why don't we talk about ways to make it just as easy, but without the requirement that indexes are downloaded locally even when they are not being used? What if we provided a reasonable message if no deb-src lines are defined, with a single simple command to add them and run apt-get update for you? From a technical point of view, does mirroring the deb lines into deb-src lines work in all cases? Would doing so break anything? -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Robie Basak [2013-05-21 13:55 +0100]: Why don't we talk about ways to make it just as easy, but without the requirement that indexes are downloaded locally even when they are not being used? pull-lp-source and pull-debian-source are about as easy as it can be IMHO, and offer a lot more functionality (downloading from different pockets, releases, and Debian). We don't install those by default, though, they are in ubuntu-dev-tools. But these are obvious candiates for replacing apt-get source in the absence of deb-src apt sources. Martin -- Martin Pitt| http://www.piware.de Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org) -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On 21 May 2013 13:55, Robie Basak robie.ba...@canonical.com wrote: What if we provided a reasonable message if no deb-src lines are defined, with a single simple command to add them and run apt-get update for you? I don't think it would even need that - software-properties (Software Updates) already has the necessary checkbox. All that is needed to enable sources is to tick that box. From a technical point of view, does mirroring the deb lines into deb-src lines work in all cases? Would doing so break anything? This is effectively what Software Sources does under-the-hood. I have to agree, if the amount being downloaded is not trivial (which I thought it was) then there's no need to have them enabled by default when it's very easy to turn them on. One of the first things I do on any new install is disable those that aren't needed. Jonathon (to the list this time) -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Am Dienstag, den 21.05.2013, 18:02 +0800 schrieb Daniel J Blueman: I propose we either disable source downloading by default at release time, but I conclude that developers generally don't care about this extra overhead (as we have a good setup). If really we can't see this from a user PoV, I'm happy to start a user discussion and see how users feel...? ubuntu-desktop ? Or where do the users discuss ? My two cents: Those who want the source will not be too blockheaded to find it later ... -- Thomas Prost t...@prosts.info ProstsInfo -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Tue, May 21, 2013 at 03:04:20PM +0100, J Fernyhough wrote: On 21 May 2013 13:55, Robie Basak robie.ba...@canonical.com wrote: What if we provided a reasonable message if no deb-src lines are defined, with a single simple command to add them and run apt-get update for you? I don't think it would even need that - software-properties (Software Updates) already has the necessary checkbox. All that is needed to enable sources is to tick that box. Provided that the user knows that the box is there. Otherwise, it risks making the availability of the source obscure, and this is where I agree with Scott in that it is against the spirit of free software to make source availability obscure. I'm not going to make a subjective judgement as to what constitutes obscurity here. I tend to edit sources.list directly, so it's not really my area. There's also the server use case to consider. We don't have the software-properties GUI, which is why I proposed the message on an apt-get source failure due to no sources being defined. From a technical point of view, does mirroring the deb lines into deb-src lines work in all cases? Would doing so break anything? This is effectively what Software Sources does under-the-hood. Perhaps we could implement enabling the sources easily from the CLI using the underlying Python library? software-properties-common and python3-software-properties are seeded on Server. Robie -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Tue, May 21, 2013 at 06:02:36PM +0800, Daniel J Blueman wrote: I propose we either disable source downloading by default at release time, but I conclude that developers generally don't care about this extra overhead (as we have a good setup). If really we can't see this from a user PoV, I'm happy to start a user discussion and see how users feel...? Source is an educational tool. Learning command line is a lesson in taking control of your own computer. Kids explore. Make sure J Random's computer is full of things to intrigue and lead a 13 year old to the power of the source. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Tue, May 21, 2013 at 9:16 AM, Dale Amon a...@vnl.com wrote: Source is an educational tool. Learning command line is a lesson in taking control of your own computer. Kids explore. Make sure J Random's computer is full of things to intrigue and lead a 13 year old to the power of the source. Personally, I use apt-get source for one thing: the little note that tells me which bzr branch I can use, instead. (There's probably a better command for that, but I don't know what it is). Using the bzr branch always works better for me. It's easier to keep track of my changes and submit them (instead of remembering when it's too late that I'm working without any kind of version control), and I can use bzr builddeb to reliably create an installable package with those changes. There are also way fewer junk files. I get the source, and that's it, instead of a bunch of different tar.gz files that are going to be entirely meaningless for someone who is just exploring the system ;) Dylan -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Tue, May 21, 2013 at 03:22:50PM +0100, Robie Basak wrote: On Tue, May 21, 2013 at 03:04:20PM +0100, J Fernyhough wrote: On 21 May 2013 13:55, Robie Basak robie.ba...@canonical.com wrote: What if we provided a reasonable message if no deb-src lines are defined, with a single simple command to add them and run apt-get update for you? I don't think it would even need that - software-properties (Software Updates) already has the necessary checkbox. All that is needed to enable sources is to tick that box. Provided that the user knows that the box is there. Otherwise, it risks making the availability of the source obscure, and this is where I agree with Scott in that it is against the spirit of free software to make source availability obscure. I'm not going to make a subjective judgement as to what constitutes obscurity here. I tend to edit sources.list directly, so it's not really my area. Yep, one of the first things I do is to manually fix whatever has been done to it and add my own files back into sources.list.d. I tend to do all my work with dselect, dpkg or apt-get tools and have my own source and binary repositories. There's also the server use case to consider. We don't have the software-properties GUI, which is why I proposed the message on an apt-get source failure due to no sources being defined. From a technical point of view, does mirroring the deb lines into deb-src lines work in all cases? Would doing so break anything? This is effectively what Software Sources does under-the-hood. Perhaps we could implement enabling the sources easily from the CLI using the underlying Python library? software-properties-common and python3-software-properties are seeded on Server. Make it all transparent. Make sure people who do not know about it can discover it and learn. Open source is about liberty in the sense that, unlike Microsoft, the Linux systems are self-teaching down to their deepest levels. It should be really easy for people to get their interest piqued and to follow up on it. That is why I absolutely *hate* non-ascii configuration files. I do virtually all my maintenance in command line via zile. I have even been known to do binary edits on occasion to bypass what some obnoxious people do with binary config data. You are not just coders. You are teachers. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Hi, I'm kind of late to this thread, but I'd like to add my opinion. Personally, I think we have to many entries in sources.list. Anyone suggesting that there is little cost to entries simply hasn't tested things. There is a very real cost to having unused entries, in bandwidth, load on a proxy/mirror and local cpu time. Thanks to 'chdist', you can quite easily test this, and in my research for LP: #1177432 (cloud-images do not have backports), I've written a simple helper for that at https://gist.github.com/smoser/5586288 . I don't want to start a flame war over what exactly should be in and what should not. What I want to suggest is that we have tools to easily let the user specify things. 'apt-add-repository' has been ubiquitous and is insanely useful. I just now noticed that bug 997371 (Create command to add multiverse and -backports to apt sources) is now marked Fix-Released, and was in quantal. I think we should continue on this trend and improve it so that apt-sources management is very easy. Then, we can turn this thread into: Subject: document apt-add-repository sources Scott On Tue, 21 May 2013, Benjamin Drung wrote: Am Dienstag, den 21.05.2013, 09:31 -0700 schrieb Dylan McCall: On Tue, May 21, 2013 at 9:16 AM, Dale Amon a...@vnl.com wrote: Source is an educational tool. Learning command line is a lesson in taking control of your own computer. Kids explore. Make sure J Random's computer is full of things to intrigue and lead a 13 year old to the power of the source. Personally, I use apt-get source for one thing: the little note that tells me which bzr branch I can use, instead. (There's probably a better command for that, but I don't know what it is). You might want to use debcheckout from the devscripts package. -- Benjamin Drung Debian Ubuntu Developer -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Am Mon, 20 May 2013 10:02:41 -0700 schrieb Benjamin Kerensa bkere...@ubuntu.com: I think in most parts of the world 4MB is trivial overhead for a user. Over here in German cheap mobile data tarrifs often get you something like a few hundered MByte/month. In some rural areas it's hard to get better than 64k/s if you don't want to setup your own radio link. 4MB every few days could quite hurt you with that. -- Privacy Indicator Unity appindicator to switch privacy settings http://www.florian-diesch.de/software/indicator-privacy/ signature.asc Description: PGP signature -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On 20 May 2013 16:09, Daniel J Blueman dan...@quora.org wrote: When installing Ubuntu, I always see the source packages enabled by default. For all the general users I install Ubuntu for (including servers), it's an utter waste of bandwidth for everyone, particularly when automatically checking once a day. This is amplified eg in schools without transparent webcaches etc. Anyone get the same feeling that we should have source packages an opt-in? Daniel -- Daniel J Blueman In theory, the only time a large download is made for a full release is the initial repo source update. From then on, only the -updates and -security package lists should be changing with any frequency, and their sizes will be much smaller. With the development release (U+1) having source repos enabled is perfectly reasonable. People running this are either developers or have enough understanding to disable their deb-src. If you are installing Ubuntu and finding this a frustration across several installations then editing the sources.list after installation (before the first sources update) might work well. You could even host a prepared version ready to wget straight to the new system. J -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
I'm talking in the context of the average user, so releases. Fine for pre-release. Updating 12.04, the universe sources are an extra 4MB (almost 25% extra) downloaded whenever a package is changed. Does it really make sense when 0.1% of people actually need this? Us developers probably care the least since we're using the latest releases and have large quota/bandwidth hard and mobile lines; general users don't know about package sources to give the feedback, so I think this is a justifiable case... On 20 May 2013 23:21, J Fernyhough j.fernyho...@gmail.com wrote: On 20 May 2013 16:09, Daniel J Blueman dan...@quora.org wrote: When installing Ubuntu, I always see the source packages enabled by default. For all the general users I install Ubuntu for (including servers), it's an utter waste of bandwidth for everyone, particularly when automatically checking once a day. This is amplified eg in schools without transparent webcaches etc. Anyone get the same feeling that we should have source packages an opt-in? Daniel -- Daniel J Blueman In theory, the only time a large download is made for a full release is the initial repo source update. From then on, only the -updates and -security package lists should be changing with any frequency, and their sizes will be much smaller. With the development release (U+1) having source repos enabled is perfectly reasonable. People running this are either developers or have enough understanding to disable their deb-src. If you are installing Ubuntu and finding this a frustration across several installations then editing the sources.list after installation (before the first sources update) might work well. You could even host a prepared version ready to wget straight to the new system. J -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss -- Daniel J Blueman -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Am Montag, den 20.05.2013, 23:09 +0800 schrieb Daniel J Blueman: When installing Ubuntu, I always see the source packages enabled by default. For all the general users I install Ubuntu for (including servers), it's an utter waste of bandwidth for everyone, particularly when automatically checking once a day. This is amplified eg in schools without transparent webcaches etc. Anyone get the same feeling that we should have source packages an opt-in? I agree that we should disable the apt-src entries by default. It's easy to enable the apt-src entries for a developer. What happens when you run apt-get source with disabled apt-src entries? -- Benjamin Drung Debian Ubuntu Developer -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Monday, May 20, 2013 06:16:53 PM Benjamin Drung wrote: Am Montag, den 20.05.2013, 23:09 +0800 schrieb Daniel J Blueman: When installing Ubuntu, I always see the source packages enabled by default. For all the general users I install Ubuntu for (including servers), it's an utter waste of bandwidth for everyone, particularly when automatically checking once a day. This is amplified eg in schools without transparent webcaches etc. Anyone get the same feeling that we should have source packages an opt-in? I agree that we should disable the apt-src entries by default. It's easy to enable the apt-src entries for a developer. What happens when you run apt-get source with disabled apt-src entries? Apt will error out that it can't find the package. I think that if we are distributing binaries, we should (perhaps must, I'm not sure) enable the source repositories in order to , as a free software distribution, provide the source that goes with the binaries we distribute. Scott K -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On 20 May 2013 17:16, Benjamin Drung bdr...@ubuntu.com wrote: What happens when you run apt-get source with disabled apt-src entries? A reasonable error message: E: You must put some 'source' URIs in your sources.list J -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Mon, May 20, 2013 at 11:19 AM, Scott Kitterman ubu...@kitterman.com wrote: Apt will error out that it can't find the package. I think that if we are distributing binaries, we should (perhaps must, I'm not sure) enable the source repositories in order to , as a free software distribution, provide the source that goes with the binaries we distribute. Required to make them available, but that doesn't mean they have to be enabled inside of apt and all the source packages are readily available via packages.ubuntu.com which means you are already complying with the GPL by making them readily available. Even with that said I'm inclined to disagree with disabling them, 4MB is trivial now days. I'm more surprised that people are more upset about 4MB than the 5% that is still claimed by the system for the system which adds up to a lot more than 4MB on some systems which on a even a small 32GB SSD is what, 1.5GB? -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Hi Daniel (2013.05.20_18:07:43_+0200) Updating 12.04, the universe sources are an extra 4MB (almost 25% extra) downloaded whenever a package is changed. The release pockets aren't changed post-release. So it's a one-time download, and a continual extra overhead on -security, -updates, and -backports. SR -- Stefano Rivera http://tumbleweed.org.za/ H: +27 21 461 1230 C: +27 72 419 8559 -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
Excerpts from Daniel J Blueman's message of 2013-05-20 08:09:19 -0700: When installing Ubuntu, I always see the source packages enabled by default. For all the general users I install Ubuntu for (including servers), it's an utter waste of bandwidth for everyone, particularly when automatically checking once a day. This is amplified eg in schools without transparent webcaches etc. Anyone get the same feeling that we should have source packages an opt-in? As an Ubuntu and Debian developer I have had them turned off for a long time now. They are, IMO, a waste of everyone's bandwidth. When I want the software, I use pull-(lp|debian)-source or bzr branch ubuntu:sourcepackage. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On May 20, 2013 8:10 AM, Daniel J Blueman dan...@quora.org wrote: When installing Ubuntu, I always see the source packages enabled by default. For all the general users I install Ubuntu for (including servers), it's an utter waste of bandwidth for everyone, particularly when automatically checking once a day. This is amplified eg in schools without transparent webcaches etc. Anyone get the same feeling that we should have source packages an opt-in? I think in most parts of the world 4MB is trivial overhead for a user. Although perhaps we should consider it since some developing nations have limited bandwidth? After all a primary reason we still ship DVD's is due to bandwidth. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Mon, May 20, 2013 at 11:25:50AM -0500, Jordon Bedwell wrote: I'm more surprised that people are more upset about 4MB than the 5% that is still claimed by the system for the system which adds up to a lot more than 4MB on some systems which on a even a small 32GB SSD is what, 1.5GB? In these days of 4TB disks, I build my disks with -m 0.01 or less. -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On Monday, May 20, 2013 11:25:50 AM Jordon Bedwell wrote: On Mon, May 20, 2013 at 11:19 AM, Scott Kitterman ubu...@kitterman.com wrote: Apt will error out that it can't find the package. I think that if we are distributing binaries, we should (perhaps must, I'm not sure) enable the source repositories in order to , as a free software distribution, provide the source that goes with the binaries we distribute. Required to make them available, but that doesn't mean they have to be enabled inside of apt and all the source packages are readily available via packages.ubuntu.com which means you are already complying with the GPL by making them readily available. Even with that said I'm inclined to disagree with disabling them, 4MB is trivial now days. I'm more surprised that people are more upset about 4MB than the 5% that is still claimed by the system for the system which adds up to a lot more than 4MB on some systems which on a even a small 32GB SSD is what, 1.5GB? I think that's legally sufficient, but not in the spirit of free software. Scott K -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
Re: Source packages appropriate by default?
On 20 May 2013 18:02, Benjamin Kerensa bkere...@ubuntu.com wrote: On May 20, 2013 8:10 AM, Daniel J Blueman dan...@quora.org wrote: When installing Ubuntu, I always see the source packages enabled by default. For all the general users I install Ubuntu for (including servers), it's an utter waste of bandwidth for everyone, particularly when automatically checking once a day. This is amplified eg in schools without transparent webcaches etc. Anyone get the same feeling that we should have source packages an opt-in? I think in most parts of the world 4MB is trivial overhead for a user. Although perhaps we should consider it since some developing nations have limited bandwidth? Here in a rural part of Wales (UK) which would not normally be considered a developing nation, my connection may run as slow as 0.5mbps (depending on the phase of the moon and the weather). A 4MB download then takes significantly more than a minute, which is not trivial when one is waiting for apt-get update to complete. Assuming I understand the implications that is. In addition I know, from the ubuntu-users list, that a surprising number of users around the world still have only dial-up speed available. I have often wondered why the sources are enabled when the average user is never going to need them. Colin -- Ubuntu-devel-discuss mailing list Ubuntu-devel-discuss@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss
