>From psarc-intern-list-request at sun.com Mon Nov 24 03:33:26 2008 >Date: Mon, 24 Nov 2008 06:31:24 -0500 >From: James Carlson <james.d.carlson at sun.com> >Subject: Re: Add sparse file support to cpio [PSARC/2008/727 Self Review] >To: Don Cragun <don.cragun at sun.com> >Cc: Garrett.Damore at sun.com, Cynthia.Eastham at sun.com, PSARC-ext at sun.com >MIME-version: 1.0 >Content-transfer-encoding: 7BIT > >Don Cragun writes: >> >4) Is there any other convention or standard for holey cpio archives in >> >common use? (E.g. something supported by Linux or star?) >> >> This was discussed while we were working on PSARC/2006/361. Some other >> archivers have a --sparse long option that converts sequences of zero >> bytes into holes. The ARC gave explicit direction to the project team >> that archivers should "preserve" existing holes, not try to reduce disk >> usage by "creating" holes. None of the other archivers (except for our >> pax) have the ability to preserve existing holes in sparse files. I >> see no reason why that advice should be true for pax, but not for cpio. > >I looked over that other case, and it seems that we discussed >'--sparse' for pax as well as the GNUish compression scheme (in the >absence of direct detection of holes), but not the cpio format itself.
Yes. > >The unanswered question here is whether there's another way to encode >holes in cpio stream format, and one that is used by other platforms. >Apart from the detection question at the source side -- I agree that >trying to "compress" files is a bit odd and certainly not the focus of >this case -- are there other ways to encode them in the output format? There are no extensible headers in cpio format. The only place to store data on where the holes go in cpio format is in the file data area. The project team had the option of storing hole information followed by the complete file contents or storing the hold information followed by the file with the holes removed. Since the cpio format only has 33 bits to store the size of the file data area, the project team chose to remove the holes to increase the size of a sparse file that can be archived in cpio format. While it is true that it would be possible to encode holes data in a slightly more compact form, it is nice to have a common format for the holes data in the ustar/pax extended header records for sparse files and in the data area in the cpio file data area. > >If there are, then should we be deliberately incompatible? The star and the recent AT&T pax archivers encode sparse files using ustar/pax format archives. The project team has not seen any other attempt to encode sparse files using cpio format. So, there is no other known cpio format that handles this case. We are not being deliberately incompatible; nothing else handles this case. - Don > >-- >James Carlson, Solaris Networking <james.d.carlson at sun.com> >Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084 >MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
