Cynthia Eastham wrote:
> Garrett D'Amore wrote:
>> I have since learned, from the project team, that this problem is
>> primarily centered around solving issues with Live Upgrade. They are
>> in a bind on timing, because they are coming up on a key deliverable
>> date. Redesign of the file format, and resetting the testing that
>> has already been done, would be detrimental to some key deliverables
>> around Live Upgrade and Solaris 10.
>>
>> Therefore, I recommend an alternate proposal, which I think might
>> satisfy most of the participants here.
>>
>> 1) Relegate the ascii_sparse and odc_sparse arguments, and the
>> associated file formats to "undocumented" status. That is, they
>> won't be documented in the man pages. This way nobody should have to
>> worry too much about dealing with these file formats portably.
>> (Neither AT&T, nor GNU, nor star, ever needs deal with them.) These
>> arguments and file formats will have "Project Private" binding.
>>
>> 2) File a contract for their use by the LU (or any other projects)
>> that need them. (At the same time, we should be giving advice to
>> these projects that they should try to convert to use pax if possible.)
>>
>> 3) Anyone else looking for sparse file support should be strongly
>> urged to use the existing pax (or star, if you prefer. I don't want
>> to get into a debate about Sun pax vs. star vs GNU tar. Its not this
>> case.)
>>
>> This would allow the project to go forward with their existing code,
>> with minimal disruption to their plans, and minimal disruption to
>> "documented" formats that external parties (star, GNU tar, AT&T) have
>> to be prepared to accept.
>>
>> -- Garrett
>>
>> Joerg Schilling wrote:
>>> Joerg.Schilling at fokus.fraunhofer.de (Joerg Schilling) wrote:
>>>
>>>
>>>>>> If there are, then should we be deliberately incompatible?
>>>>>>
>>>>> The star and the recent AT&T pax archivers encode sparse files using
>>>>> ustar/pax format archives. The project team has not seen any other
>>>>> attempt to encode sparse files using cpio format. So, there is no
>>>>> other known cpio format that handles this case. We are not being
>>>>> deliberately incompatible; nothing else handles this case.
>>>>>
>>>> If you mark the cpio archives using the proposal from above and if you
>>>> use comma separated data_offset/data_size pairs to encode the hole
>>>> list,
>>>> I would be willing to implement this in star too.
>>>>
>>>
>>> Let me make a completely new proposal that does not touch any POSIX
>>> defined field and that allows to add further extensions in the
>>> future. It is based on
>>> extension ideas from David Korn and Glenn Fowler.
>>>
>>> If the archive type name is either "ascii_sparse" or "odc_sparse",
>>> the string
>>>
>>> "VSUN\0\0" is appended to all file names in the cpio header and the
>>> field c_namesize is filled with a number that is 6 bigger than
>>> expected for a standard
>>> cpio archive.
>>>
>>> If a sparse file is encountered, the following string is used instead:
>>>
>>> "VSUN\0S%jx\0\0", (uintmax_t)statbuf.st_size
>>>
>>> c_namesize is filled with a number that is of the apropriate size
>>> bigger than
>>> in a standard cpio archive.
>>>
>>> In case of a sparse file, the following hole list is used inside the
>>> file data cunk:
>>>
>>> "%jx\n%s", (uintmax_t)number_of_holes_in_list
>>>
>>> The "%s" string is replaced by "number_of_holes_in_list" entris of
>>> the following form:
>>>
>>> "%jx %jx\n", (uintmax_t)data_offset, (uintmax_t)data_size
>>>
>>> Sparse files that end in a hole are treated the same was as in star.
>>>
>>>
>>> Using hex numbers instead of decimal numbers is aligned with the
>>> AT&T format.
>>> Using hex numbers instead of decimal numbers reduces the size of the
>>> hole list by 20%. Using "data_size" instead of "hole_offset" reduces
>>> the hole list by another 20%.
>>>
>>> The field c_filesize contains a size that is equal to the compressed
>>> file size + the size of the hole list.
>>>
>>> Please comment!
>>>
>>> J?rg
>>>
>>>
>>
>
> I've talked with Don Cragun, who is off-line today for medical
> reasons, about Garrett's proposal as well as Joerg's proposal. Due to
> extreme time constraints, we are happy to make the proposed interfaces
> undocumented and Project Private. I'll work with the LU team to draw
> up a contract.
>
> In the future, but not part of this case, after discussing it with
> AT&T, my intent will be to provide Committed interfaces for handling
> sparse files along the lines of Joerg's proposal.
This sounds like an excellent way forward.
Thanks Cynthia.
-- Garrett
>
> Cindy
