On Wed, 6 Mar 2002, Eric Engelhard wrote:
> Does anyone on the list have experience using perl read
> (FILEHANDLE,SCALAR,LENGTH,OFFSET) to drop into compressed files (.Z and
> .gz) and extract readable results? I am ignorant of both compressed file
> anatomy and/or perl mods which can handle this issue.
> I currently process large reference files (up to 1.8GB compressed)
> through a decompression/recompression pipe, but would like to divide and
> conquer on parallel cluster nodes without having to read through the
> entire file at each node.
I don't think you can do seeks in a compressed file... you have to read it
sequentially.
If you have a plan for dividing up the uncompressed data, perhaps you
should do that first and store the split data as separate files
(recompressed or not) for purposes of computation.
---------------------------------------------------------------------------
Jeff Newmiller The ..... ..... Go Live...
DCN:<[EMAIL PROTECTED]> Basics: ##.#. ##.#. Live Go...
Live: OO#.. Dead: OO#.. Playing
Research Engineer (Solar/Batteries O.O#. #.O#. with
/Software/Embedded Controllers) .OO#. .OO#. rocks...2k
---------------------------------------------------------------------------
_______________________________________________
vox-tech mailing list
[EMAIL PROTECTED]
http://lists.lugod.org/mailman/listinfo/vox-tech
