Marcin Owsiany <[EMAIL PROTECTED]> writes:
> On Sun, Feb 04, 2007 at 06:49:37PM +0000, Joe Wells wrote:
>> I agree that cruft uses too much disk space.
>>
>> For example, on my system, "du -s /var/spool/cruft" says 45940 (KiB),
>> but after running "gzip /var/spool/cruft/*" the output of "du -s
>> /var/spool/cruft" changes to 4252, which is only 9% of the
>> uncompressed size.
>
> This sounds like a clever idea.. If you feel like it, could you please
> do some size and time change measurments for this data, for different
> compression levels (say, --fast, default and --best)?
Sure. Output included below. Quick summary: -9 (same as --best) gets
a compressed result that is 8.5% of the uncompressed size. -1 (same
as --fast) gets a compressed result that is 10.3% of the uncompressed
size. The -9 compressed result is 83% of the size of the -1
compressed result. The default (same as -6) compressed size is 9% of
the uncompressed size. Compressing with -9 takes 460% of the time
used to compress with -1 and 280% of the time used to compress with
-6. The compression time with -9 is only 6 seconds though so it is
probably much smaller than the time taken to gather the data being
compressed. Uncompression time seems to depend only on the compressed
size, so it did not vary much. bzip2 compresses better (7.4% of
uncompressed size) but takes a *lot* longer to compress and
uncompress. (bzip2 also uses tons of memory when compressing and
uncompressing.)
Here are the stats:
uncompressed size: 45760120
size compressed with "gzip -9": 3909493
size compressed with "gzip -1": 4723511
size compressed with "gzip -6": 4108964
size compressed with "bzip2": 3380903
time to compress with "gzip -9": 6.17u + 0.09s = 6.26
time to compress with "gzip -1": 1.26u + 0.09s = 1.35
time to compress with "gzip -6": 2.15u + 0.08s = 2.23
time to compress with "bzip2": 26.83u + 0.18s = 27.01
time to uncompress with "gzip -9": 0.49u + 0.16s = 0.65
time to uncompress with "gzip -1": 0.52u + 0.16s = 0.68
time to uncompress with "gzip -6": 0.48u + 0.17s = 0.65
time to uncompress with "bzip2": 3.41u + 0.21s = 3.62
>> How hard would it be to change cruft to save its files in compressed
>> (e.g., maybe using gzip) format?
>
> Saving would be trivial - just filtering data through gzip in a shell
> script. However generating a report from such files would not be so
> straightforward, as an important part of it is done by a small program
> written in C, which opens the files on its own, so this would require
> adding zlib support to it.
>
> I'm not saying that it's impossible, but I'm currently looking at
> reimplementing some of the cruft's guts, so it's going to wait at least
> until I finish that.
Okay. Hope it can be done, because it would be really nice to save
all that space (my disks are always nearly full!).
--
Joe Wells
======================================================================
Script started on 2007-02-05 T 15:47:02
> sudo time --verbose gzip -9 *
Command being timed: "gzip -9 expl_alternatives expl_dev
expl_diversions expl_dpkg expl_users file_in_dev file_in_dev-pts
file_in_dev-shm file_in_home file_in_mnt-shared-data2 file_in_proc-bus-usb
file_in_sys file_in_var-lock file_in_var-run file_root miss_alternatives
miss_dev miss_diversions miss_dpkg miss_users need_alternatives need_link_dests
report unex_in_home unex_in_mnt-shared-data2 unex_in_proc-bus-usb unex_in_sys
unex_in_var-lock unex_in_var-run unex_root want_alternatives want_link_dests"
User time (seconds): 6.17
System time (seconds): 0.09
Percent of CPU this job got: 93%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:06.67
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 215
Voluntary context switches: 1
Involuntary context switches: 6353
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
> gzip -l *
compressed uncompressed ratio uncompressed_name
1458 8928 84.1% expl_alternatives
4046 25080 84.0% expl_dev
349 1992 84.2% expl_diversions
847072 8510446 90.0% expl_dpkg
944074 12772141 92.6% expl_users
86 179 70.9% file_in_dev-pts
45 9 -22.2% file_in_dev-shm
2465 12114 79.9% file_in_dev
941235 12755631 92.6% file_in_home
833 9286 91.5% file_in_mnt-shared-data2
99 258 76.7% file_in_proc-bus-usb
21455 214812 90.0% file_in_sys
71 50 28.0% file_in_var-lock
312 907 69.3% file_in_var-run
903342 9035497 90.0% file_root
107 95 25.3% miss_alternatives
1387 12796 89.4% miss_dev
239 1215 83.1% miss_diversions
511 3048 84.2% miss_dpkg
82 95 44.2% miss_users
748 3779 81.2% need_alternatives
48503 407126 88.1% need_link_dests
97358 1081703 91.0% report
325 904 67.5% unex_in_home
831 9268 91.5% unex_in_mnt-shared-data2
99 258 76.7% unex_in_proc-bus-usb
21453 214807 90.0% unex_in_sys
59 22 -9.1% unex_in_var-lock
294 810 67.9% unex_in_var-run
54888 501895 89.1% unex_root
71 36 2.8% want_alternatives
15596 174933 91.1% want_link_dests
3909493 45760120 91.5% (totals)
> sudo time --verbose gunzip *
Command being timed: "gunzip expl_alternatives.gz expl_dev.gz
expl_diversions.gz expl_dpkg.gz expl_users.gz file_in_dev-pts.gz
file_in_dev-shm.gz file_in_dev.gz file_in_home.gz file_in_mnt-shared-data2.gz
file_in_proc-bus-usb.gz file_in_sys.gz file_in_var-lock.gz file_in_var-run.gz
file_root.gz miss_alternatives.gz miss_dev.gz miss_diversions.gz miss_dpkg.gz
miss_users.gz need_alternatives.gz need_link_dests.gz report.gz unex_in_home.gz
unex_in_mnt-shared-data2.gz unex_in_proc-bus-usb.gz unex_in_sys.gz
unex_in_var-lock.gz unex_in_var-run.gz unex_root.gz want_alternatives.gz
want_link_dests.gz"
User time (seconds): 0.49
System time (seconds): 0.16
Percent of CPU this job got: 88%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.74
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 173
Voluntary context switches: 1
Involuntary context switches: 672
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
> sudo time --verbose gzip -1 *
Command being timed: "gzip -1 expl_alternatives expl_dev
expl_diversions expl_dpkg expl_users file_in_dev file_in_dev-pts
file_in_dev-shm file_in_home file_in_mnt-shared-data2 file_in_proc-bus-usb
file_in_sys file_in_var-lock file_in_var-run file_root miss_alternatives
miss_dev miss_diversions miss_dpkg miss_users need_alternatives need_link_dests
report unex_in_home unex_in_mnt-shared-data2 unex_in_proc-bus-usb unex_in_sys
unex_in_var-lock unex_in_var-run unex_root want_alternatives want_link_dests"
User time (seconds): 1.26
System time (seconds): 0.09
Percent of CPU this job got: 93%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.45
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 217
Voluntary context switches: 1
Involuntary context switches: 1386
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
> gzip -l *
compressed uncompressed ratio uncompressed_name
1661 8928 81.8% expl_alternatives
4695 25080 81.4% expl_dev
412 1992 81.0% expl_diversions
1042721 8510446 87.7% expl_dpkg
1120052 12772141 91.2% expl_users
87 179 70.4% file_in_dev-pts
45 9 -22.2% file_in_dev-shm
2885 12114 76.4% file_in_dev
1116679 12755631 91.2% file_in_home
1003 9286 89.7% file_in_mnt-shared-data2
103 258 75.2% file_in_proc-bus-usb
25651 214812 88.1% file_in_sys
71 50 28.0% file_in_var-lock
332 907 67.1% file_in_var-run
1112710 9035497 87.7% file_root
107 95 25.3% miss_alternatives
1607 12796 87.7% miss_dev
277 1215 80.0% miss_diversions
613 3048 80.8% miss_dpkg
82 95 44.2% miss_users
818 3779 79.3% need_alternatives
58538 407126 85.6% need_link_dests
118862 1081703 89.0% report
331 904 66.8% unex_in_home
995 9268 89.7% unex_in_mnt-shared-data2
103 258 75.2% unex_in_proc-bus-usb
25643 214807 88.1% unex_in_sys
59 22 -9.1% unex_in_var-lock
315 810 65.3% unex_in_var-run
67241 501895 86.6% unex_root
71 36 2.8% want_alternatives
18742 174933 89.3% want_link_dests
4723511 45760120 89.7% (totals)
> sudo time --verbose gunzip *
Command being timed: "gunzip expl_alternatives.gz expl_dev.gz
expl_diversions.gz expl_dpkg.gz expl_users.gz file_in_dev-pts.gz
file_in_dev-shm.gz file_in_dev.gz file_in_home.gz file_in_mnt-shared-data2.gz
file_in_proc-bus-usb.gz file_in_sys.gz file_in_var-lock.gz file_in_var-run.gz
file_root.gz miss_alternatives.gz miss_dev.gz miss_diversions.gz miss_dpkg.gz
miss_users.gz need_alternatives.gz need_link_dests.gz report.gz unex_in_home.gz
unex_in_mnt-shared-data2.gz unex_in_proc-bus-usb.gz unex_in_sys.gz
unex_in_var-lock.gz unex_in_var-run.gz unex_root.gz want_alternatives.gz
want_link_dests.gz"
User time (seconds): 0.52
System time (seconds): 0.16
Percent of CPU this job got: 89%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.76
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 174
Voluntary context switches: 1
Involuntary context switches: 710
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
> sudo time --verbose gzip *
Command being timed: "gzip expl_alternatives expl_dev expl_diversions
expl_dpkg expl_users file_in_dev file_in_dev-pts file_in_dev-shm file_in_home
file_in_mnt-shared-data2 file_in_proc-bus-usb file_in_sys file_in_var-lock
file_in_var-run file_root miss_alternatives miss_dev miss_diversions miss_dpkg
miss_users need_alternatives need_link_dests report unex_in_home
unex_in_mnt-shared-data2 unex_in_proc-bus-usb unex_in_sys unex_in_var-lock
unex_in_var-run unex_root want_alternatives want_link_dests"
User time (seconds): 2.15
System time (seconds): 0.08
Percent of CPU this job got: 92%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.43
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 217
Voluntary context switches: 1
Involuntary context switches: 2249
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
> gzip -l *
compressed uncompressed ratio uncompressed_name
1498 8928 83.6% expl_alternatives
4235 25080 83.2% expl_dev
350 1992 84.1% expl_diversions
890947 8510446 89.5% expl_dpkg
989479 12772141 92.3% expl_users
86 179 70.9% file_in_dev-pts
45 9 -22.2% file_in_dev-shm
2486 12114 79.7% file_in_dev
986484 12755631 92.3% file_in_home
943 9286 90.3% file_in_mnt-shared-data2
99 258 76.7% file_in_proc-bus-usb
22579 214812 89.5% file_in_sys
71 50 28.0% file_in_var-lock
310 907 69.6% file_in_var-run
952426 9035497 89.5% file_root
107 95 25.3% miss_alternatives
1542 12796 88.2% miss_dev
240 1215 83.0% miss_diversions
512 3048 84.1% miss_dpkg
82 95 44.2% miss_users
753 3779 81.0% need_alternatives
50052 407126 87.7% need_link_dests
104366 1081703 90.4% report
325 904 67.5% unex_in_home
938 9268 90.3% unex_in_mnt-shared-data2
99 258 76.7% unex_in_proc-bus-usb
22576 214807 89.5% unex_in_sys
59 22 -9.1% unex_in_var-lock
293 810 68.0% unex_in_var-run
58314 501895 88.4% unex_root
71 36 2.8% want_alternatives
16597 174933 90.5% want_link_dests
4108964 45760120 91.0% (totals)
> sudo time --verbose gunzip *
Command being timed: "gunzip expl_alternatives.gz expl_dev.gz
expl_diversions.gz expl_dpkg.gz expl_users.gz file_in_dev-pts.gz
file_in_dev-shm.gz file_in_dev.gz file_in_home.gz file_in_mnt-shared-data2.gz
file_in_proc-bus-usb.gz file_in_sys.gz file_in_var-lock.gz file_in_var-run.gz
file_root.gz miss_alternatives.gz miss_dev.gz miss_diversions.gz miss_dpkg.gz
miss_users.gz need_alternatives.gz need_link_dests.gz report.gz unex_in_home.gz
unex_in_mnt-shared-data2.gz unex_in_proc-bus-usb.gz unex_in_sys.gz
unex_in_var-lock.gz unex_in_var-run.gz unex_root.gz want_alternatives.gz
want_link_dests.gz"
User time (seconds): 0.48
System time (seconds): 0.17
Percent of CPU this job got: 88%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.74
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 172
Voluntary context switches: 2
Involuntary context switches: 663
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
> sudo time --verbose bzip2 -v *
expl_alternatives: 6.498:1, 1.231 bits/byte, 84.61% saved, 8928 in,
1374 out.
expl_dev: 7.632:1, 1.048 bits/byte, 86.90% saved, 25080 in,
3286 out.
expl_diversions: 5.298:1, 1.510 bits/byte, 81.12% saved, 1992 in,
376 out.
expl_dpkg: 11.138:1, 0.718 bits/byte, 91.02% saved, 8510446
in, 764122 out.
expl_users: 16.365:1, 0.489 bits/byte, 93.89% saved, 12772141
in, 780470 out.
file_in_dev: 5.802:1, 1.379 bits/byte, 82.76% saved, 12114 in,
2088 out.
file_in_dev-pts: 2.157:1, 3.709 bits/byte, 53.63% saved, 179 in,
83 out.
file_in_dev-shm: 0.184:1, 43.556 bits/byte, -444.44% saved, 9 in,
49 out.
file_in_home: 16.388:1, 0.488 bits/byte, 93.90% saved, 12755631
in, 778375 out.
file_in_mnt-shared-data2: 12.498:1, 0.640 bits/byte, 92.00% saved, 9286 in,
743 out.
file_in_proc-bus-usb: 2.745:1, 2.915 bits/byte, 63.57% saved, 258 in,
94 out.
file_in_sys: 13.765:1, 0.581 bits/byte, 92.74% saved, 214812
in, 15606 out.
file_in_var-lock: 0.667:1, 12.000 bits/byte, -50.00% saved, 50 in,
75 out.
file_in_var-run: 2.660:1, 3.008 bits/byte, 62.40% saved, 907 in,
341 out.
file_root: 11.130:1, 0.719 bits/byte, 91.02% saved, 9035497
in, 811819 out.
miss_alternatives: 0.856:1, 9.347 bits/byte, -16.84% saved, 95 in,
111 out.
miss_dev: 11.274:1, 0.710 bits/byte, 91.13% saved, 12796 in,
1135 out.
miss_diversions: 4.483:1, 1.784 bits/byte, 77.70% saved, 1215 in,
271 out.
miss_dpkg: 5.655:1, 1.415 bits/byte, 82.32% saved, 3048 in,
539 out.
miss_users: 1.105:1, 7.242 bits/byte, 9.47% saved, 95 in, 86
out.
need_alternatives: 4.694:1, 1.704 bits/byte, 78.70% saved, 3779 in,
805 out.
need_link_dests: 8.842:1, 0.905 bits/byte, 88.69% saved, 407126
in, 46046 out.
report: 12.195:1, 0.656 bits/byte, 91.80% saved, 1081703
in, 88699 out.
unex_in_home: 2.354:1, 3.398 bits/byte, 57.52% saved, 904 in,
384 out.
unex_in_mnt-shared-data2: 12.872:1, 0.621 bits/byte, 92.23% saved, 9268 in,
720 out.
unex_in_proc-bus-usb: 2.745:1, 2.915 bits/byte, 63.57% saved, 258 in,
94 out.
unex_in_sys: 13.783:1, 0.580 bits/byte, 92.74% saved, 214807
in, 15585 out.
unex_in_var-lock: 0.355:1, 22.545 bits/byte, -181.82% saved, 22 in,
62 out.
unex_in_var-run: 2.477:1, 3.230 bits/byte, 59.63% saved, 810 in,
327 out.
unex_root: 9.718:1, 0.823 bits/byte, 89.71% saved, 501895
in, 51644 out.
want_alternatives: 0.500:1, 16.000 bits/byte, -100.00% saved, 36 in,
72 out.
want_link_dests: 11.343:1, 0.705 bits/byte, 91.18% saved, 174933
in, 15422 out.
Command being timed: "bzip2 -v expl_alternatives expl_dev
expl_diversions expl_dpkg expl_users file_in_dev file_in_dev-pts
file_in_dev-shm file_in_home file_in_mnt-shared-data2 file_in_proc-bus-usb
file_in_sys file_in_var-lock file_in_var-run file_root miss_alternatives
miss_dev miss_diversions miss_dpkg miss_users need_alternatives need_link_dests
report unex_in_home unex_in_mnt-shared-data2 unex_in_proc-bus-usb unex_in_sys
unex_in_var-lock unex_in_var-run unex_root want_alternatives want_link_dests"
User time (seconds): 26.83
System time (seconds): 0.18
Percent of CPU this job got: 94%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:28.65
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 11610
Voluntary context switches: 1
Involuntary context switches: 28080
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
> du -s
3424 .
> sudo time --verbose bunzip2 *
Command being timed: "bunzip2 expl_alternatives.bz2 expl_dev.bz2
expl_diversions.bz2 expl_dpkg.bz2 expl_users.bz2 file_in_dev-pts.bz2
file_in_dev-shm.bz2 file_in_dev.bz2 file_in_home.bz2
file_in_mnt-shared-data2.bz2 file_in_proc-bus-usb.bz2 file_in_sys.bz2
file_in_var-lock.bz2 file_in_var-run.bz2 file_root.bz2 miss_alternatives.bz2
miss_dev.bz2 miss_diversions.bz2 miss_dpkg.bz2 miss_users.bz2
need_alternatives.bz2 need_link_dests.bz2 report.bz2 unex_in_home.bz2
unex_in_mnt-shared-data2.bz2 unex_in_proc-bus-usb.bz2 unex_in_sys.bz2
unex_in_var-lock.bz2 unex_in_var-run.bz2 unex_root.bz2 want_alternatives.bz2
want_link_dests.bz2"
User time (seconds): 3.41
System time (seconds): 0.21
Percent of CPU this job got: 56%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:06.40
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 6219
Voluntary context switches: 1
Involuntary context switches: 3681
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
> exit
Script done on 2007-02-05 T 15:55:14
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
