On Fri, Jun 13, 2008 at 6:30 AM, Neil Graham <[EMAIL PROTECTED]> wrote:
> As it stands now there seems to be no 100% reliable way to judge the
> compressed size of things on jffs2.
>
> I cast my eye to the boot-anim, uncompressed it comes to about 60 Meg, is
> there space being wasted there?
>
> http://lists.laptop.org/pipermail/library/2007-July/000070.html says
>> JFFS2 compresses 4K chunks using zlib. So it's not just per-file
>> compression, it's compressing bits of a file. It doesn't compress files
>> where compression doesn't help. And there's a 68 byte overhead per
>> compressed chunk. Plus probably some fixed overhead per file.
>
> doing some tests using mkfs.jffs2 on a filesystem with a single file of zeros
> at varying sizes gave me.
>
> 409600 --> 11656
> 819200 --> 23256
> 1228800 --> 34856
> 1638400 --> 46456
>
> each growth of 100 4k blocks added 11600 bytes. I'll go out on a limb and say
> the best case on jffs2 is turning a 4k block into 116 bytes, about 35:1
> compression, or down to 2.8% if you prefer
>
> This is why the boot-anim bothers me. If it were all zeros it should be
> taking up just over 1.5 meg. It compresses really well, but it can't go
> beyond that limit.
>
> Running the jffs2size.py script on /usr/share/boot-anim reveals.
>
> no compression : 60480336, 60480K
> estimated jffs2: 1983630, 1983K (3%)
> mkfs.jffs2 : 2344068, 2344K (3%)
> zipped : 389259, 389K (0%)
> .tar.gz : 399360, 399K (0%)
> .tar.bz2 : 184320, 184K (0%)
>
> It looks like there's easily a meg to be had there, two meg looks doable.
>
> Where to go from here is another issue. total flexibility and 90% of the size
> gain could be had by using pngs (not sure if there is a speed issue there)
>
> If there were to be a simple runlengh encoding, the bulk of the size would go
> and jffs2 would crunch the remainder. wadeb whipped up a tiny rle
> compressor/decompressor pair.
>
> gzipping at a file level removes the block overhead and the size drops to
> about 380k.
>
> For smallest size, lzma compressed rle files. appear to be the best. lzma
> isn't in the default install. It is small at 90k and has potential other user
> applications.
>
> 6325 frame00.565.gz
> 2283 frame00.565.lzma
> 15236 frame00.565.rle
> 1841 frame00.565.rle.gz
> 1451 frame00.565.rle.lzma
>
> 28097 ul_warning.565.lzma
> 30594 ul_warning.565.rle.gz
> 22935 ul_warning.565.rle.lzma
>
> The final compression method is colour-of-the-bikeshed stuff, the important
> thing here is freeing up the 2 meg. Whether the actual saving can be1.9MB or
> 2.2MB is less important.
Thank you for the thorough quantitative data! Nothing helps convince
me more than numbers. =)
With the attached 4-line patch to ppmto565.py, we generate
bzip2-compressed files with a total (non-jffs2) size of 98,939 bytes.
mkfs.jffs2 size is 107,176 bytes.
By changing two lines (see attached gzip.path), we generate
zlib-compressed files. Total non-jffs size: 210,096 bytes (!).
mkfs.jffs2 size: 177,544 bytes.
I suspect zlib is available in the initramfs, but bzip is probably not
(and lzma is definitely not). Even though it costs 70k, I think I'd
go with zlib here.
I'm about to leave for 10 days vacation; bonus points to anyone who
can take the 4 line change to ppmto565.py and turn this into a 'real'
patch for 8.2. Source code is at:
http://dev.laptop.org/git?p=users/cscott/act-gui;a=summary
Tasks are:
a) patch fbutil.c and/or pyfb.pyx to properly load both compressed
and uncompressed images (since some countries have customized the boot
sequence and fielded .565 files we should continue to support:
http://wiki.laptop.org/go/Tweaking_the_boot_animation )
b) patch boot-anim-start and/or pyfb.pyx to allow *un*loading boot
animation frames. It doesn't matter when the frames are straight
mmaps from disk, because the kernel knows it can through those pages
out of the page cache if it needs extra memory. When you're doing
computation on the mmapped data, then the pages are dirty and get
stuck in memory, so we probably need to free the uncompressed frames
we're no longer using.
c) double check that this doesn't have a significant performance
impact on the XO, probably by writing a simple script which loads &
displays the images similar to pytest.py and timing the script on an
XO both before and after the changes.
For extra credit, you could insert your code into the initramfs by
rebuilding it as described at the bottom of
http://wiki.laptop.org/go/Kernel_Building, and double check that it
doesn't break (ie, that we really do have zlib in the initramfs). If
we don't, then it's probably reasonable to leave the activation and
first frame parts of the animation in uncompressed form, and only
compress frames 01-25, which are displayed after we've left the
initramfs.
Happy hacking!
--scott
--
( http://cscott.net/ )
--- ppmto565.py 2007-08-04 21:13:03.000000000 -0400
+++ ppmto565z.py 2008-06-13 10:16:07.000000000 -0400
@@ -26,6 +26,7 @@
def main(inf, outf):
import struct
+ from bz2 import BZ2Compressor
# read ppm from input
magic = inf.read(2)
assert magic == 'P6'
@@ -35,7 +36,8 @@
unpackstr = '!3B' if maxval < 256 else '!3H'
unpacklen = struct.calcsize(unpackstr)
# write header
- outf.write(struct.pack('@IIP', width, height, 0))
+ outf.write(struct.pack('@IIP', width, height, 1))
+ comp = BZ2Compressor(9)
# read pixel data, convert, and write it to output
def scale(x, y): return int((y*x/maxval)+0.5)
for y in xrange(0, height):
@@ -44,7 +46,8 @@
encoded = (scale(red, 0x1f) << 11) + \
(scale(green, 0x3f) << 5) + \
(scale(blue, 0x1f))
- outf.write(struct.pack('@H', encoded))
+ outf.write(comp.compress(struct.pack('@H', encoded)))
+ outf.write(comp.flush())
if __name__ == '__main__':
import sys
--- ppmto565.py 2007-08-04 21:13:03.000000000 -0400
+++ ppmto565z.py 2008-06-13 10:41:29.000000000 -0400
@@ -26,6 +26,7 @@
def main(inf, outf):
import struct
+ from zlib import compressobj
# read ppm from input
magic = inf.read(2)
assert magic == 'P6'
@@ -35,7 +36,8 @@
unpackstr = '!3B' if maxval < 256 else '!3H'
unpacklen = struct.calcsize(unpackstr)
# write header
- outf.write(struct.pack('@IIP', width, height, 0))
+ outf.write(struct.pack('@IIP', width, height, 1))
+ comp = compressobj(9)
# read pixel data, convert, and write it to output
def scale(x, y): return int((y*x/maxval)+0.5)
for y in xrange(0, height):
@@ -44,7 +46,8 @@
encoded = (scale(red, 0x1f) << 11) + \
(scale(green, 0x3f) << 5) + \
(scale(blue, 0x1f))
- outf.write(struct.pack('@H', encoded))
+ outf.write(comp.compress(struct.pack('@H', encoded)))
+ outf.write(comp.flush())
if __name__ == '__main__':
import sys
_______________________________________________
Devel mailing list
Devel@lists.laptop.org
http://lists.laptop.org/listinfo/devel
