Re: How about using bzip2 as the standard *.deb compression format?
On Thu, Oct 08, 1998 at 06:40:09AM -0500, John Goerzen wrote: > dpkg remains the primary bottleneck in the setup, and apt calls dpkg > anyway, so the different is not really significant, and apt-get update > is slow too. The update phase seems to be slow because of translating the package files to dselect's format on my system. (pending apt's GUI, I'm using dselect) If dselect could use apt's native format, that would be faster. If package files were made by section, apt would not have to download main's pacakages.gz everytime one little package in one little section was updated. dpkg too would benefit from using a hashed database for lookups too, though I think a text database that could be rehased would be nice too. If others think that's not necessary, I'll live without it. After all, dpkg is quite stable despite a couple buglets. If the new database code were also stable (read: simplistic enough that it would just work) I'd not worry about dpkg losing its database or anything. I'm curious as to what enhancements Ian has planned for dpkg whenever he finds time to return to working on it. pgpwMhJmr2xQ6.pgp Description: PGP signature
Re: How about using bzip2 as the standard *.deb compression format?
dpkg remains the primary bottleneck in the setup, and apt calls dpkg anyway, so the different is not really significant, and apt-get update is slow too. Joseph Carter <[EMAIL PROTECTED]> writes: > [1 ] > On Tue, Oct 06, 1998 at 03:50:01PM -0500, John Goerzen wrote: > > This is silly. dpkg/dselect are already insanely slow, even on my > > P166 with 128 meg of RAM -- especially when reading database, etc. If > > we slow down the installation so much more by using bzip2, then people > > will simply stop upgrading, or switch to other distributions because > > it is so slow. That is not acceptable. > > Um, not all of us are using dselect/dpkg. Most of us refuse to because it's > insanely slow and generally braindead if you have a serious conflict. I use > dselect/apt myself. > [2 ] > -- John Goerzen Linux, Unix consulting & programming [EMAIL PROTECTED] | Developer, Debian GNU/Linux (Free powerful OS upgrade) www.debian.org | + Visit the Air Capital Linux Users Group on the web at http://www.aclug.org
Re: How about using bzip2 as the standard *.deb compression format?
On Tue, 06 Oct, 1998, Joseph Carter wrote: > On Tue, Oct 06, 1998 at 03:50:01PM -0500, John Goerzen wrote: > > This is silly. dpkg/dselect are already insanely slow, even on my > > P166 with 128 meg of RAM -- especially when reading database, etc. If > > we slow down the installation so much more by using bzip2, then people > > will simply stop upgrading, or switch to other distributions because > > it is so slow. That is not acceptable. > > Um, not all of us are using dselect/dpkg. Most of us refuse to because it's > insanely slow and generally braindead if you have a serious conflict. I use > dselect/apt myself. surley you mean dselect/apt/dpkg? -- linux: because a PC is a terrible thing to waste
Re: How about using bzip2 as the standard *.deb compression format?
Joseph Carter wrote: > I doubt it would compile on my 4 meg 486. > > Nor would it run there. I've ran X on 2 mb. (shoot me.. please.. ;-) -- see shy jo
Re: How about using bzip2 as the standard *.deb compression format?
On Tue, Oct 06, 1998 at 03:50:01PM -0500, John Goerzen wrote: > This is silly. dpkg/dselect are already insanely slow, even on my > P166 with 128 meg of RAM -- especially when reading database, etc. If > we slow down the installation so much more by using bzip2, then people > will simply stop upgrading, or switch to other distributions because > it is so slow. That is not acceptable. Um, not all of us are using dselect/dpkg. Most of us refuse to because it's insanely slow and generally braindead if you have a serious conflict. I use dselect/apt myself. pgpNPlhjmG8xs.pgp Description: PGP signature
Re: How about using bzip2 as the standard *.deb compression format?
On 6 Oct 1998, John Goerzen wrote: > This is silly. dpkg/dselect are already insanely slow, even on my > P166 with 128 meg of RAM -- especially when reading database, etc. If > we slow down the installation so much more by using bzip2, then people > will simply stop upgrading, or switch to other distributions because > it is so slow. That is not acceptable. The new bzip2 0.9 (?), while not as fast as gzip, is considerably faster than the older bzip2 0.1pl2 (??) (I can't remember the version numbers. :-) Considering the time saved during downloading, I would say that the use of bzip2 would save time overall for most people. Anthony <[EMAIL PROTECTED]>
Re: How about using bzip2 as the standard *.deb compression format?
Right now I'm using a 200MMX with 64MB, which used to be a 133MHz with 64MB and I always found the speed of dpkg perfectly acceptable. Are you using the outdated dselect method that scans every single package every time you install one little component, and do have like 400 packages installed with 60,000 files on your disk? Please fully consider all the points I made in my other emails. Christopher John Goerzen wrote: > > This is silly. dpkg/dselect are already insanely slow, even on my > P166 with 128 meg of RAM -- especially when reading database, etc. If > we slow down the installation so much more by using bzip2, then people > will simply stop upgrading, or switch to other distributions because > it is so slow. That is not acceptable. > > John > > Christopher Barry <[EMAIL PROTECTED]> writes: > > > If your mighty 386/25 with 4MB can make World the entire X distribution > > and custom kernels then surely it won't sweat a little bit of bzip2 > > decompressing... and since you spend a lot less time downloading a > > bzip2ed *.deb, the extra time bzip2 would take by swapping and thrashing > > the disk should balance out nicely. > > > > Christopher > > > > > > James Troup wrote: > > > > > > Joseph Carter <[EMAIL PROTECTED]> writes in gratuitous QP: > > > > > > > On Sun, Oct 04, 1998 at 12:15:40PM +0100, James Troup wrote: > > > > > > Old/slow/lomem machines can't properly compile X or Mozilla anyway. > > > > > > > > > > Bzzt. I've compiled xfree86 for Debian/m68k on a 386/25 equivalent > > > > > with only 14Mb (don't ask) of memory several times. Took 5 days, > > > > > like, but it compiled ``properly''. > > > > > > > > I doubt it would compile on my 4 meg 486. > > > > > > I don't; I compiled kernels on the same machine when it only had 4Mb. > > > > > > > Nor would it run there. > > > > > > And I know it ran on my Falcon with 4Mb... > > > > > > -- > > > James > > > > > > -- > > > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > > > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] > > > > > > -- > > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] > > > > -- > John Goerzen Linux, Unix consulting & programming [EMAIL PROTECTED] | > Developer, Debian GNU/Linux (Free powerful OS upgrade) www.debian.org | > + > Visit the Air Capital Linux Users Group on the web at http://www.aclug.org
Re: How about using bzip2 as the standard *.deb compression format?
This is silly. dpkg/dselect are already insanely slow, even on my P166 with 128 meg of RAM -- especially when reading database, etc. If we slow down the installation so much more by using bzip2, then people will simply stop upgrading, or switch to other distributions because it is so slow. That is not acceptable. John Christopher Barry <[EMAIL PROTECTED]> writes: > If your mighty 386/25 with 4MB can make World the entire X distribution > and custom kernels then surely it won't sweat a little bit of bzip2 > decompressing... and since you spend a lot less time downloading a > bzip2ed *.deb, the extra time bzip2 would take by swapping and thrashing > the disk should balance out nicely. > > Christopher > > > James Troup wrote: > > > > Joseph Carter <[EMAIL PROTECTED]> writes in gratuitous QP: > > > > > On Sun, Oct 04, 1998 at 12:15:40PM +0100, James Troup wrote: > > > > > Old/slow/lomem machines can't properly compile X or Mozilla anyway. > > > > > > > > Bzzt. I've compiled xfree86 for Debian/m68k on a 386/25 equivalent > > > > with only 14Mb (don't ask) of memory several times. Took 5 days, > > > > like, but it compiled ``properly''. > > > > > > I doubt it would compile on my 4 meg 486. > > > > I don't; I compiled kernels on the same machine when it only had 4Mb. > > > > > Nor would it run there. > > > > And I know it ran on my Falcon with 4Mb... > > > > -- > > James > > > > -- > > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] > -- John Goerzen Linux, Unix consulting & programming [EMAIL PROTECTED] | Developer, Debian GNU/Linux (Free powerful OS upgrade) www.debian.org | + Visit the Air Capital Linux Users Group on the web at http://www.aclug.org
Re: How about using bzip2 as the standard *.deb compression format?
On Mon 05 Oct 1998, Paul Slootman wrote: > On Sun 04 Oct 1998, James Troup wrote: > > Joseph Carter <[EMAIL PROTECTED]> writes: > > > > > Old/slow/lomem machines can't properly compile X or Mozilla anyway. > > > > Bzzt. I've compiled xfree86 for Debian/m68k on a 386/25 equivalent > > with only 14Mb (don't ask) of memory several times. Took 5 days, > > 14MB isn't that lomem... BTW, I just had a look at the new bzip2 version. This are the relevant lines from top while running 'bz2cat x': PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND 30413 paul 20 0 6820 6820 288 R 0 72.0 10.7 3:55 bzip2 30412 paul 0 0 3928 3928 288 S 0 23.5 6.2 0:48 bz2cat Decompressing doesn't take that much time nor memory, if I compare it for example with my X server: 265 root 0 0 15028 11M 1004 S 0 0.5 19.0 542:35 XF86_SVGA Of course, 4MB is still quite a lot, but I guess that should be doable for just about everyone. Alternatively, from the manpage: Compression and decompression requirements, in bytes, can be estimated as: Compression: 400k + ( 7 x block size ) Decompression: 100k + ( 4 x block size ), or 100k + ( 2.5 x block size ) and For files compressed with the default 900k block size, bunzip2 will require about 3700 kbytes to decompress. To support decompression of any file on a 4 megabyte machine, bunzip2 has an option to decompress using approximately half this amount of memory, about 2300 kbytes. Decompres sion speed is also halved, so you should use this option only where necessary. The relevant flag is -s. So, I think that some experimentation of what block sizes and flags to use may be in order. Besides, as decompression is done internally by dpkg (right?), dpkg could check the memory available on the machine and decide which decompression algorithm to use. In short, I don't really think that there are compelling arguments _not_ to consider bzip2. And yes, x ended up identical to linux-2.1.124.tar.bz2 in case you're wondering :-) Paul Slootman -- home: [EMAIL PROTECTED] | work: [EMAIL PROTECTED] | debian: [EMAIL PROTECTED] http://www.wurtel.demon.nl | Murphy Software, Enschede, the Netherlands
Re: How about using bzip2 as the standard *.deb compression format?
It seems the whole point you are making is that there is nothing old, slow, lowmem machines can't handle that should be bzip2 compressed, but my point is that if there is no package that a slow, lomem machine can't handle, like an X or Emacs source distribution, then the slow, lowmem machine could handle a demanding decompressor to. You've got to think about majority rule and what benefits the most users overall. I brought up the bzip2 thing initially because it was mentioned that the main Debian distribution will no longer be able to fit on a CD, and media and shipping costs could be nicely reduced if we could get the whole thing back onto a single CD again, and also buy more time in getting the package management system to deal with more than one CD. Additionally, for users that don't have an ethernet connection to a T1 but like to keep their system up to date with Apt, which I think is the category most people running Debian on their home box fall into, it's nice to have much faster downloads. Especially for people with metered internet access like a lot of ISDN plans. The point is, anyone with a P5-60 or faster machine gains nothing but benefit from moving to bzip2, and the poeple stuck with older machines will still be able to use bzip2 and if their net connection is slow, the extra time bzip2 takes decompressing may even balance out. I'm sure people spending a great deal of time on old slow boxes running Debian are a very small minority, and they can make the small sacrifice of longer decompression times so that all of the benefits mentioned above like reduced media and shipping costs, more time to get multi-cd support for package management, reduced download times, reduced monthly bills for those with metered access, reduced disk space usage on ftp servers, reduced load on ftp mirrors, reduced usage of local disk space for those of us that like to keep a local mirror, etc., etc Christopher James Troup wrote: > > Christopher Barry <[EMAIL PROTECTED]> writes: > > > If your mighty 386/25 > ^ > > a) cut out the sarcasm, it's uncalled for. > > b) get your facts right, it's not a 386, it's a 386/25 equivalent[1] >as I said already. > > > with 4MB can make World the entire X distribution and custom kernels > > then surely it won't sweat a little bit of bzip2 decompressing... > > I didn't say it wouldn't; I was trying to point out what complete > rubbish "Old/slow/lomem machines can't properly compile X or Mozilla > anyway." was. > > I'm not interested in the bzip2 discussion per se, because it seems > like your average Debian discussion, with lots of people ra-ra-ing but > no danger of anyone actually getting down and doing any real work. > > > and since you spend a lot less time downloading a bzip2ed *.deb, > > That depends entirely on one's network connection. The time saved on > my network connection for the previous 3 years wouldn't have actually > been measurable. > > > the extra time bzip2 would take by swapping and thrashing the disk > > should balance out nicely. > > IYO and IYE. Mileage does vary. > > [1] It's actually a [EMAIL PROTECTED] with the mother of all brain dead > motherboard designs which slows it down by a factor of 2 or so. As > you can see, I'm not overly proud of the machine, quite the opposite > in fact. > > -- > James > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: How about using bzip2 as the standard *.deb compression format?
On Sun 04 Oct 1998, James Troup wrote: > Joseph Carter <[EMAIL PROTECTED]> writes: > > > Old/slow/lomem machines can't properly compile X or Mozilla anyway. > > Bzzt. I've compiled xfree86 for Debian/m68k on a 386/25 equivalent > with only 14Mb (don't ask) of memory several times. Took 5 days, 14MB isn't that lomem... Paul Slootman -- home: [EMAIL PROTECTED] | work: [EMAIL PROTECTED] | debian: [EMAIL PROTECTED] http://www.wurtel.demon.nl | Murphy Software, Enschede, the Netherlands
Re: How about using bzip2 as the standard *.deb compression format?
Christopher Barry <[EMAIL PROTECTED]> writes: > If your mighty 386/25 ^ a) cut out the sarcasm, it's uncalled for. b) get your facts right, it's not a 386, it's a 386/25 equivalent[1] as I said already. > with 4MB can make World the entire X distribution and custom kernels > then surely it won't sweat a little bit of bzip2 decompressing... I didn't say it wouldn't; I was trying to point out what complete rubbish "Old/slow/lomem machines can't properly compile X or Mozilla anyway." was. I'm not interested in the bzip2 discussion per se, because it seems like your average Debian discussion, with lots of people ra-ra-ing but no danger of anyone actually getting down and doing any real work. > and since you spend a lot less time downloading a bzip2ed *.deb, That depends entirely on one's network connection. The time saved on my network connection for the previous 3 years wouldn't have actually been measurable. > the extra time bzip2 would take by swapping and thrashing the disk > should balance out nicely. IYO and IYE. Mileage does vary. [1] It's actually a [EMAIL PROTECTED] with the mother of all brain dead motherboard designs which slows it down by a factor of 2 or so. As you can see, I'm not overly proud of the machine, quite the opposite in fact. -- James
Re: How about using bzip2 as the standard *.deb compression format?
If your mighty 386/25 with 4MB can make World the entire X distribution and custom kernels then surely it won't sweat a little bit of bzip2 decompressing... and since you spend a lot less time downloading a bzip2ed *.deb, the extra time bzip2 would take by swapping and thrashing the disk should balance out nicely. Christopher James Troup wrote: > > Joseph Carter <[EMAIL PROTECTED]> writes in gratuitous QP: > > > On Sun, Oct 04, 1998 at 12:15:40PM +0100, James Troup wrote: > > > > Old/slow/lomem machines can't properly compile X or Mozilla anyway. > > > > > > Bzzt. I've compiled xfree86 for Debian/m68k on a 386/25 equivalent > > > with only 14Mb (don't ask) of memory several times. Took 5 days, > > > like, but it compiled ``properly''. > > > > I doubt it would compile on my 4 meg 486. > > I don't; I compiled kernels on the same machine when it only had 4Mb. > > > Nor would it run there. > > And I know it ran on my Falcon with 4Mb... > > -- > James > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: How about using bzip2 as the standard *.deb compression format?
Joseph Carter <[EMAIL PROTECTED]> writes in gratuitous QP: > On Sun, Oct 04, 1998 at 12:15:40PM +0100, James Troup wrote: > > > Old/slow/lomem machines can't properly compile X or Mozilla anyway. > > > > Bzzt. I've compiled xfree86 for Debian/m68k on a 386/25 equivalent > > with only 14Mb (don't ask) of memory several times. Took 5 days, > > like, but it compiled ``properly''. > > I doubt it would compile on my 4 meg 486. I don't; I compiled kernels on the same machine when it only had 4Mb. > Nor would it run there. And I know it ran on my Falcon with 4Mb... -- James
Re: How about using bzip2 as the standard *.deb compression format?
On Sun, Oct 04, 1998 at 12:15:40PM +0100, James Troup wrote: > > Old/slow/lomem machines can't properly compile X or Mozilla anyway. > > Bzzt. I've compiled xfree86 for Debian/m68k on a 386/25 equivalent > with only 14Mb (don't ask) of memory several times. Took 5 days, > like, but it compiled ``properly''. I doubt it would compile on my 4 meg 486. Nor would it run there. pgpn0ZgJGOcE9.pgp Description: PGP signature
Re: How about using bzip2 as the standard *.deb compression format?
Joseph Carter <[EMAIL PROTECTED]> writes: > Old/slow/lomem machines can't properly compile X or Mozilla anyway. Bzzt. I've compiled xfree86 for Debian/m68k on a 386/25 equivalent with only 14Mb (don't ask) of memory several times. Took 5 days, like, but it compiled ``properly''. -- James
Re: How about using bzip2 as the standard *.deb compression format?
On Sat, Oct 03, 1998 at 08:37:12AM -0500, dsb3 wrote: > >> I think we already went through this discussion a short while back. > >> Unless I'm missing something new, it was pretty much decided that the > >> memory overhead of bzip2 was too great for low-mem or slow PCs to handle. > > > >It'd STILL be nice to be able to use bzip2 for package source on REALLY BIG > >packages (Mozilla, X) > > very good point! those users with slow / low mem machines are less likely > to be installing these packages anyway! Perhaps we could compromise by > saying that anyone running these on a slow machine will be patient anyway > and can deal with the extra slowness and disk thrashing of using bzip2? Old/slow/lomem machines can't properly compile X or Mozilla anyway. pgp0G9gISxbWZ.pgp Description: PGP signature
Re: How about using bzip2 as the standard *.deb compression format?
On Fri, 2 Oct 1998, Joseph Carter wrote: >> (I said) >> I think we already went through this discussion a short while back. >> Unless I'm missing something new, it was pretty much decided that the >> memory overhead of bzip2 was too great for low-mem or slow PCs to handle. >> > >It'd STILL be nice to be able to use bzip2 for package source on REALLY BIG >packages (Mozilla, X) > very good point! those users with slow / low mem machines are less likely to be installing these packages anyway! Perhaps we could compromise by saying that anyone running these on a slow machine will be patient anyway and can deal with the extra slowness and disk thrashing of using bzip2? - dave -- | oOOooO / --|oOobodoO/ [EMAIL PROTECTED] --| ooOoOo / | II / "Rocky Road," croaked the toad. | II /
Re: How about using bzip2 as the standard *.deb compression format?
On Fri, 2 October 1998 22:25:35 -0700, Joseph Carter wrote: > It'd STILL be nice to be able to use bzip2 for package source on REALLY BIG > packages (Mozilla, X) I agree. It'd be fine for now if it's supported and then you can still decide to use it for your own packages. You won't install X or mozilla on boxes with, say, 4 megs of RAM, right? ;-) Just a thought. Alexander -- - Real programmers don't document. Documentation is for simps who can't read the listings of the object deck. Alexander Koch - <>< - aka Efraim - PGP - 0xE7694969 - Hannover - Germany pgpB7yk5907Kn.pgp Description: PGP signature
Re: How about using bzip2 as the standard *.deb compression format?
On Fri, Oct 02, 1998 at 10:06:24PM -0500, dsb3 wrote: > >I read in an earlier mail that the main distro will no longer fit on one > >CD. Since a standardised specialized tool is already required to install > >a *.deb and this tool is installed on every Debian box, why not in the > >next update of dpkg include support to decompress bzip2 compressed > >*.debs? This would be transparent for the user, and (as far as I can > >reason anyways) fairly painless for the developer. > > I think we already went through this discussion a short while back. > Unless I'm missing something new, it was pretty much decided that the > memory overhead of bzip2 was too great for low-mem or slow PCs to handle. > > That said, please correct me if I got the wrong end of the stick. It'd STILL be nice to be able to use bzip2 for package source on REALLY BIG packages (Mozilla, X) pgpXFkKSO57Fv.pgp Description: PGP signature
Re: How about using bzip2 as the standard *.deb compression format?
On Fri, 2 Oct 1998, Christopher Barry wrote: >Hi, > >I read in an earlier mail that the main distro will no longer fit on one >CD. Since a standardised specialized tool is already required to install >a *.deb and this tool is installed on every Debian box, why not in the >next update of dpkg include support to decompress bzip2 compressed >*.debs? This would be transparent for the user, and (as far as I can >reason anyways) fairly painless for the developer. > I think we already went through this discussion a short while back. Unless I'm missing something new, it was pretty much decided that the memory overhead of bzip2 was too great for low-mem or slow PCs to handle. That said, please correct me if I got the wrong end of the stick. - dave -- | oOOooO / --|oOobodoO/ [EMAIL PROTECTED] --| ooOoOo / | II / "Rocky Road," croaked the toad. | II /
