On Wed, Apr 30, 2014 at 6:33 AM, roberto.mine...@usi.ch < roberto.mine...@usi.ch> wrote:
> Hi, > > Following the Fuel test FLUserGuidesTest>>#testGZip I discovered that I > can serialize and materialize to and from GZip archives. Cool! > > I tried with some small test object and it worked like a charm. When I > tried with a large object… > > Yes, I have the exact same problem long ago. As far as I could debug at that point in time, it was a bug in the compression plugin in the VM. So there isn't anything easy you can do about. See http://lists.gforge.inria.fr/pipermail/pharo-project/2011-December/056419.html for more details. > Serialization > - It does not work, the image instantly crashes > > Materialization > - Since I was not able to serialize the large object on GZip, I serialized > "normally" than compressed the resulting fuel file using GZipWriteStream > class>>#compressFile: > - When I materialize from the GZip archive, it takes forever and does not > succeed after more than 1 minute. Note that the same non-Gzipped file takes > a couple of seconds to be materialized. > > If you serialized normally and the compressed, then to materialize you need to first uncompress and then give fuel the decompressed version. BTW...what I did for Fuel and compressing is a Pharo binding to a really fast compressor/uncompressor: LZ4. http://smalltalkhub.com/#!/~marianopeck/LZ4 That is very fast, but it doesn't provide a streaming API. So the usage I did was: - Fuel serialize in memory to a bytearray - LZ4 compress that bytearray and write compressed to file or whatever you want - LZ4 uncompress from file or from where you stored the compressed version. - Fuel materialize from the compressed stream from previous step this will work even for very large graphs. There is also an experiment Max and I were doing about compressing strings right inside Fuel: https://code.google.com/p/fuel/issues/detail?id=160 But this is committed in the experiments repo. If you want to take a look... Best, -- Mariano http://marianopeck.wordpress.com
