Compression on the other hand might be helpful to avoid precompressing everything beforehand.
I fear that is going to be pretty slow and will eat at least 1.5 the memory of the file you are trying to store.
If you want a good compression ratio. then again... it might be fast enough to still be useful.
