[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006969#comment-16006969 ] Zbynek Vyskovsky commented on COMPRESS-391: --- [~bodewig] : Thank you. Looks like I still missed few magic numbers and the style too, thanks for the update. > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006936#comment-16006936 ] ASF GitHub Bot commented on COMPRESS-391: - Github user asfgit closed the pull request at: https://github.com/apache/commons-compress/pull/24 > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16006910#comment-16006910 ] Stefan Bodewig commented on COMPRESS-391: - Many thanks [~kvr], I'll merge your PR tonight with few minor (mostly formatting) tweaks added as a separate commit. I didn't expect the pkware folks to e that fast, great. > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005832#comment-16005832 ] Zbynek Vyskovsky commented on COMPRESS-391: --- I updated the pull request with the RFC update. Now it both reads and writes alignment and adjust padding according to current offset. In some terms it's actually simpler than the previous solutions and still provides more complex functionality. It also stores requested alignment details into both local and central directory but padding is added only into local directory. Could you please check and let me know your thoughts? > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005830#comment-16005830 ] ASF GitHub Bot commented on COMPRESS-391: - Github user coveralls commented on the issue: https://github.com/apache/commons-compress/pull/24 [](https://coveralls.io/builds/11469394) Coverage increased (+0.09%) to 84.309% when pulling **77f5ce3714e194704332aa66bca746c9806f2be7 on kvr000:feature/COMPRESS-391-allow-entries-alignment** into **ca7ea939eaa9dad5f00c8a5e269f09681602bc98 on apache:master**. > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005301#comment-16005301 ] Zbynek Vyskovsky commented on COMPRESS-391: --- The guys from Zip were quite quick, it's already published in preliminary version: https://support.pkware.com/display/PKZIP/Proposed+ZIP+Format+Specification+Additions So probably worth sticking to this extra field and implement both reading and writing... What do you think? > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[
https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004988#comment-16004988
]
Zbynek Vyskovsky commented on COMPRESS-391:
---
I thought it zipalign aligns .so files to page boundary but maybe I forgot
details.
With the latest changes I'd only afraid that padding can only grow, maybe we
can reset (subtract the existing padding) before we calculate new one (only in
case the alignment check failed, as you implemented it).
Extra field 4.6.10 could work if we don't want to store the alignment, good
catch.
Anyway, I sent the proposal so let see if it's approved, we may then modify the
solution:
{code}
4.6.11 -Data Stream Alignment (0xa11e):
Defines alignment of data stream of this entry within the zip
archive. Additionally, indicates whether the compression
method should be kept when re-compressing the zip file.
The purpose of this extra field is to align specific resources
to word or page boundary so they can be easily mapped into
memory.
Value SizeDescription
- ---
0xa11eShort tag for this extra block type
TSize Short total data size for this block (2+padding)
alignment Short required alignment and indicator
0x00 Variablepadding
The alignment field (lower 15 bits) defines the minimal alignment
required by the data stream.
Bit 15 of alignment field indicates whether compression method
of this entry can be changed when recompressing the zip file.
0 means the compression method should be kept, 1 indicates
the compression method can be changed.
The padding field contains padding to ensure the correct alignment
and can be changed at any time when the offset changes or required
alignment changes.
{code}
> Zip entries alignment
> -
>
> Key: COMPRESS-391
> URL: https://issues.apache.org/jira/browse/COMPRESS-391
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.13
>Reporter: Zbynek Vyskovsky
> Labels: features, github-import, patch
> Fix For: 1.14
>
>
> Similarly to COMPRESS-390, there are requirements of the zip content to be
> mapped directly into memory and therefore may require special alignment on
> the embedded files. E.g. libraries may be aligned to page (4096-bytes)
> boundary, images on 4-bytes boundary etc. By alignment it's meant the offset
> from the beginning of file where the actual data stream starts, not the
> header.
> One of the cases was (still is?) Android APK for which zipalign utility was
> created.
> It would be useful if commons-compress ZipArchiveOutputStream supports
> something similar directly in its API.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[
https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004285#comment-16004285
]
Stefan Bodewig commented on COMPRESS-391:
-
I've pushed additional commits on the COMPRESS-391 branch. Feel free to ignore
what I've done. Really.
Reading is implemented as well ({{parseFromLocalFileData}} reads the padding
length).
I've installed zipalign and it always aligns at 4 byte boundaries (and does so
for all stored entries). I wonder whether alignment should be a property of
{{ZipArchiveOutputStream}} in addition to {{ZipArchiveEntry}} in order to apply
it to all stored entries as well. You are correct, zipalign simply adds zeros
which ends up in an {{UnparseableExtraField}} when we read it.
As for proposing the extra field, it won't hurt, I'm just not sure how quick
the response will be (and {{l}} is not a valid hex character ;-) 0xa11e?). Hmm,
take a look at "4.6.10 -Microsoft Open Packaging Growth Hint (0xa220)" this
looks as if we could use it, I couldn't find any information what its real
purpose would be. Of course it doesn't store the alignment value you propose.
> Zip entries alignment
> -
>
> Key: COMPRESS-391
> URL: https://issues.apache.org/jira/browse/COMPRESS-391
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.13
>Reporter: Zbynek Vyskovsky
> Labels: features, github-import, patch
> Fix For: 1.14
>
>
> Similarly to COMPRESS-390, there are requirements of the zip content to be
> mapped directly into memory and therefore may require special alignment on
> the embedded files. E.g. libraries may be aligned to page (4096-bytes)
> boundary, images on 4-bytes boundary etc. By alignment it's meant the offset
> from the beginning of file where the actual data stream starts, not the
> header.
> One of the cases was (still is?) Android APK for which zipalign utility was
> created.
> It would be useful if commons-compress ZipArchiveOutputStream supports
> something similar directly in its API.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004052#comment-16004052 ] Zbynek Vyskovsky commented on COMPRESS-391: --- [~bodewig] : Oh, I should be faster in implementing the patches if I don't want to wait for another release :-) Thanks for this! I think first the field can be removed completely and then only added if the alignment doesn't match. Although with approach you mentioned the copying of alignment would still work as long as the old files streams and structure remains exactly the same... which is kind of fragile though... I believe it would still make sense to write the requested alignment value (not padding value) as part of the payload, it's just two bytes anyway and it may be used later when someone implements both reading and writing part. Additionally, in such case this extra field should be written always, not only when alignment doesn't match, as it contains important information for re-zipping. About zipalign: As far as I remember, they simply add enough zeroes at the end of extra fields, they even don't create the id header, its size etc. So the result is not really correct zip stream but I assume the readers simply ignore it. So no "prior art" here... About Zip RFC: According to https://support.pkware.com/display/PKZIP/APPNOTE , there is an e-mail for proposing enhancements. I can write and update here. 0xalle (id), 0x0002 (size), short (alignment) sounds good? :-) > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[
https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004029#comment-16004029
]
Stefan Bodewig commented on COMPRESS-391:
-
[~kvr] don't worry. I'm eager to get 1.14 out, that's why I started to
implenent something myself.
bq. I think it might make sense to take the length from extra once re-populated
True, but for a different reason. {{addExtraField}} replaces an existing extra
field using the same id, so if there has already been an extra field by that id
the length of extra would be different. It will be cleaner to try removing an
existing padding field (or adjusting, I'm bad at names :-), recalculate the
size, add one, and maybe recalculate the size again. I'll modify the patch
accordingly. This should address your concern about previously existing
alignment and having only one instance of the extra field.
The field is really only useful when writing the archive, so I don't think we
need to perform any special step when alignment changes. This should be
addressed when the entry with the changed alignment is written (again) -
something that should happen when I modify the existing code.
WRT the id I understood we would mimic "prior art" used by zipalign. We should
use the same id it uses. As for Zip RFC, I'm not sure about the process, or if
there is a process at all.
> Zip entries alignment
> -
>
> Key: COMPRESS-391
> URL: https://issues.apache.org/jira/browse/COMPRESS-391
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.13
>Reporter: Zbynek Vyskovsky
> Labels: features, github-import, patch
> Fix For: 1.14
>
>
> Similarly to COMPRESS-390, there are requirements of the zip content to be
> mapped directly into memory and therefore may require special alignment on
> the embedded files. E.g. libraries may be aligned to page (4096-bytes)
> boundary, images on 4-bytes boundary etc. By alignment it's meant the offset
> from the beginning of file where the actual data stream starts, not the
> header.
> One of the cases was (still is?) Android APK for which zipalign utility was
> created.
> It would be useful if commons-compress ZipArchiveOutputStream supports
> something similar directly in its API.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002836#comment-16002836 ] Zbynek Vyskovsky commented on COMPRESS-391: --- [~bodewig] : Looks like it should work, I think it might make sense to take the length from extra once re-populated but that's fine. I'm also wondering what happens when someone copies the zip stream, it would copy this padding to the output (and potentially add new one if requested), right? That wouldn't make too much sense as the purpose of this field is not really padding but alignment of the stream. When you suggested it, I started thinking more about self-maintaining solution. First step is the same, i.e. creating extra field called FileAlignment (I prefer the name alignment because it better expresses its purpose). That would be encoded as variable length depending on the required padding but two bytes would be mandatory, containing the requested alignment. Therefore it would be possible to read the previously requested alignment from existing zip file and pass it to another one. It would also make sense to assign it some reasonable id as it got some meaning (and maybe send request to add to Zip RFC). The class would still require special handling as there should be just single instance and whenever setAlignment() is called we would have to replace the old instance (actually we wouldn't need setAlignment anymore really but maybe it could be useful if added to common interface and making it reusable across other implementations). As well as it would require special handling during serialization - it should be ignored first when constructing extra fields array, then padding should be added and only then should be appended to extra field. Because of this I feel the original code was actually better as it required special handling only at one place. What are your thoughts? And Sorry I haven't responded earlier, it's more complex and I haven't found time to look at this yet. > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002215#comment-16002215 ] ASF GitHub Bot commented on COMPRESS-391: - Github user bodewig commented on the issue: https://github.com/apache/commons-compress/pull/24 @kvr000 what do you think about https://github.com/apache/commons-compress/commit/620196621e15a87cdd5e3382504bd8a9829f4698 ? > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998420#comment-15998420 ] ASF GitHub Bot commented on COMPRESS-391: - Github user bodewig commented on the issue: https://github.com/apache/commons-compress/pull/24 I don't see a chance of making it independent of `ZipArchiveOutputStream`, you are certainly correct. I was thinking along the lines of `ZipArchiveOutputStream` calculates the length needed for proper alignment, creates an instance of the new extra field passing in the size information necessary and adds it to the `ZipArchiveEntry` - after that the code in `ZipArchiveOutputStream` can be left unchanged. > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998399#comment-15998399 ] ASF GitHub Bot commented on COMPRESS-391: - Github user kvr000 commented on the issue: https://github.com/apache/commons-compress/pull/24 @bodewig : Thanks for suggestions. There are definitely some pros with the approach, especially if it reaches Zip RFC :-) On the other hand, I don't think it could be made completely independent on ZipArchiveOutputStream as it has to know current stream offset on which to base the alignment. But I'll check in details... > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997998#comment-15997998 ] ASF GitHub Bot commented on COMPRESS-391: - Github user bodewig commented on the issue: https://github.com/apache/commons-compress/pull/24 Many thanks @kvr000 Have you thought about adding a class implementing `ZipExtraField` for padding? You could add that to the entry if you detect padding is necessary and could remove code that now needs to know about the encoding of extra fields from `ZipArchiveOutputStream`. The central directory size should be 0 then. It would also preserve the padding information when an archive is read which may be useful when re-packing existing entries as you could strip or modify the padding extra field directly. Finally I wonder whether `setAlignment` should reject values bigger than 64k right away. > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997763#comment-15997763 ] ASF GitHub Bot commented on COMPRESS-391: - Github user coveralls commented on the issue: https://github.com/apache/commons-compress/pull/24 [](https://coveralls.io/builds/11383081) Coverage increased (+0.05%) to 84.279% when pulling **e42d33b01848cd46c87739bf92c33a6c0f136f32 on kvr000:feature/COMPRESS-391-allow-entries-alignment** into **092bcac5be680480860dbddae4217347b8b14fab on apache:master**. > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997761#comment-15997761 ] ASF GitHub Bot commented on COMPRESS-391: - GitHub user kvr000 opened a pull request: https://github.com/apache/commons-compress/pull/24 COMPRESS-391: Allow alignment on zip content You can merge this pull request into a Git repository by running: $ git pull https://github.com/kvr000/commons-compress feature/COMPRESS-391-allow-entries-alignment Alternatively you can review and apply these changes as the patch at: https://github.com/apache/commons-compress/pull/24.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #24 commit e42d33b01848cd46c87739bf92c33a6c0f136f32 Author: Zbynek Vyskovsky Date: 2017-05-05T03:44:44Z COMPRESS-391: Allow alignment on zip content > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985980#comment-15985980 ] Stefan Bodewig commented on COMPRESS-391: - Ah, I somehow thought the idea was to pad the length of the entry, not its data offset. Thanks. > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985326#comment-15985326 ] Zbynek Vyskovsky commented on COMPRESS-391: --- [~bodewig] : I updated the description. In this context it means the offset in the file where the data starts. For example we may want to map the embedded library and since the file has to be mapped on page (4096b) boundary and we need to map the library with the same restriction, it must align on the page boundary within the zip file. About how to achieve it: I already have implementation but waiting until COMPRESS-390 reaches master as we can better verify in the unit tests (you can check at https://github.com/kvr000/commons-compress/commits/feature/COMPRESS-391-allow-entries-alignment). But briefly, yes, extra fields is used for that - we calculate the offset where the header would end (and would data start), calculate the number of bytes for padding to adjust to the alignment boundary (potentially adding additional 4 as the size of extra field header). Then we create dummy extra field (0x) with length of padding-4 (extra field header size). > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. E.g. libraries may be aligned to page (4096-bytes) > boundary, images on 4-bytes boundary etc. By alignment it's meant the offset > from the beginning of file where the actual data stream starts, not the > header. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (COMPRESS-391) Zip entries alignment
[ https://issues.apache.org/jira/browse/COMPRESS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985261#comment-15985261 ] Stefan Bodewig commented on COMPRESS-391: - Have you got any details what "alignment" means? Do you know how it is achieved? I can imagine using zip extra fields for padding but that won't work unless you know the deflated size upfront. > Zip entries alignment > - > > Key: COMPRESS-391 > URL: https://issues.apache.org/jira/browse/COMPRESS-391 > Project: Commons Compress > Issue Type: New Feature > Components: Archivers >Affects Versions: 1.13 >Reporter: Zbynek Vyskovsky > Labels: features, github-import, patch > Fix For: 1.14 > > > Similarly to COMPRESS-390, there are requirements of the zip content to be > mapped directly into memory and therefore may require special alignment on > the embedded files. > One of the cases was (still is?) Android APK for which zipalign utility was > created. > It would be useful if commons-compress ZipArchiveOutputStream supports > something similar directly in its API. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
