[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542595#comment-16542595
]
Stefan Bodewig commented on COMPRESS-459:
-
Not immediately unless somebody else cuts the release (upcoming vacation for
me). I want to add a proper test for COMPRESS-455 before cutting a new release
which will be 1.18 when I'm back.
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
> Labels: patch-available
> Fix For: 1.18
>
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541742#comment-16541742
]
Jens Reimann commented on COMPRESS-459:
---
Thanks for taking care of this.
Is there any chance for a soon-ish 1.17.1 version?
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
> Labels: patch-available
> Fix For: 1.18
>
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540368#comment-16540368
]
ASF GitHub Bot commented on COMPRESS-459:
-
Github user asfgit closed the pull request at:
https://github.com/apache/commons-compress/pull/67
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
> Labels: patch-available
> Fix For: 1.18
>
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540369#comment-16540369
]
ASF GitHub Bot commented on COMPRESS-459:
-
Github user bodewig commented on the issue:
https://github.com/apache/commons-compress/pull/67
Many thanks!
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
> Labels: patch-available
> Fix For: 1.18
>
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540294#comment-16540294
]
ASF GitHub Bot commented on COMPRESS-459:
-
Github user bodewig commented on the issue:
https://github.com/apache/commons-compress/pull/67
@ctron I've just started merging your changes locally
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
> Labels: patch-available
> Fix For: 1.18
>
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540049#comment-16540049
]
ASF GitHub Bot commented on COMPRESS-459:
-
Github user bodewig commented on the issue:
https://github.com/apache/commons-compress/pull/67
`ZipEncoding` is a better choice for our internal use as it is used to
encode the name (and deals with "use null as the encoding to use the platform's
default encoding" transparently). You don't have to make the changes yourself
(but if you do, please add spaces around operators :-) ), but I won't complain
if you are faster than me.
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
> Labels: patch-available
> Fix For: 1.18
>
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539852#comment-16539852
]
ASF GitHub Bot commented on COMPRESS-459:
-
Github user ctron commented on the issue:
https://github.com/apache/commons-compress/pull/67
Yes, that might have been a solution as well. But I was thinking to use
something more Java-standard. If you want me to change this, I can do this as
well.
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Archivers
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
> Labels: patch-available
> Fix For: 1.18
>
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539723#comment-16539723
]
ASF GitHub Bot commented on COMPRESS-459:
-
Github user bodewig commented on the issue:
https://github.com/apache/commons-compress/pull/67
Many thanks. We are not that squash-happy over here, two commits is fine.
I would have gone for `ZipEncoding` rather than `Charset` for consistency,
but can change that later myself, I really don't want to play "fetch me a rock"
here. Will merge the PR soonish.
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Compressors
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539652#comment-16539652
]
ASF GitHub Bot commented on COMPRESS-459:
-
Github user ctron commented on the issue:
https://github.com/apache/commons-compress/pull/67
Thanks @bodewig for reviewing this PR. Yes you are right, with everything
:wink:
I amended the PR, fixing the issues with the original PR and added the
functionality to the output stream as well. The made them two commits for to
easier understand what was fixed and what added. It can be squashed later on if
required.
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Compressors
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16539618#comment-16539618
]
ASF GitHub Bot commented on COMPRESS-459:
-
Github user bodewig commented on the issue:
https://github.com/apache/commons-compress/pull/67
Thanks @ctron!
Don't we need to adjust `getHeaderPadCount` in `readOldBinaryEntry` as well?
At first I stumbled over the `-1` in the call to `getHeaderPadCount` but I
think it is because the method already accounts for the trailing NUL, I guess a
comment could help :-)
One more nit, you removed a since tag from one of `CpioArchiveEntry`'s
constructors by accident.
Also, but this probably is a separate issue: `CpioArchiveOutputStream`
seems to have the same problem (even worse, as the length we write into the
header may be wrong). It seems we should deprecate the no-arg version of
`getHeaderPadCount` as it only works for single bye encodings.
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Compressors
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16536739#comment-16536739
]
ASF GitHub Bot commented on COMPRESS-459:
-
Github user coveralls commented on the issue:
https://github.com/apache/commons-compress/pull/67
[](https://coveralls.io/builds/17891202)
Coverage increased (+0.002%) to 86.355% when pulling
**715352d3343358539a40b9623a9a00beb115ff30 on
ctron:feature/fix_COMPRESS_459_1** into
**f5330f7e667f5a7245c8a5f3007cda04554c5fe2 on apache:master**.
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Compressors
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (COMPRESS-459) CPIO fails decoding multibyte name entries
[
https://issues.apache.org/jira/browse/COMPRESS-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16536735#comment-16536735
]
ASF GitHub Bot commented on COMPRESS-459:
-
GitHub user ctron opened a pull request:
https://github.com/apache/commons-compress/pull/67
[COMPRESS-459] Fix reading of multibyte name entries
This fixes COMPRESS-459 by using the name number of bytes from the field
in the stream instead of relying on the assumption that each character
is exactly one byte, which isn't true for UTF-8, UTF-16 or other
multi-byte character encodings.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ctron/commons-compress
feature/fix_COMPRESS_459_1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/commons-compress/pull/67.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #67
commit 715352d3343358539a40b9623a9a00beb115ff30
Author: Jens Reimann
Date: 2018-07-09T09:41:43Z
[COMPRESS-459] Fix reading of multibyte name entries
This fixes COMPRESS-459 by using the name number of bytes from the field
in the stream instead of relying on the assumption that each character
is exactly one byte, which isn't true for UTF-8, UTF-16 or other
multi-byte character encodings.
> CPIO fails decoding multibyte name entries
> --
>
> Key: COMPRESS-459
> URL: https://issues.apache.org/jira/browse/COMPRESS-459
> Project: Commons Compress
> Issue Type: Bug
> Components: Compressors
>Affects Versions: 1.9, 1.17
>Reporter: Jens Reimann
>Priority: Major
>
> Having a CPIO archive in (e.g. UTF-8) mode and having a name entry with a
> name containing multi-byte characters the decoder fails.
> The problem IMHO is the "getHeaderPadCount" method, which assumes a single
> byte per character:
>
> {code:java}
> public int getHeaderPadCount(){
> if (this.alignmentBoundary == 0) { return 0; }
> int size = this.headerSize + 1; // Name has terminating null
> if (name != null) {
> size += name.length();
> }
> final int remain = size % this.alignmentBoundary;
> if (remain > 0){
> return this.alignmentBoundary - remain;
> }
> return 0;
> }
> {code}
> However this may (or may not) be true for UTF-8.
>
> Also it wouldn't be enough to call "String#getBytes(…)" as this might already
> transform the underlying bytes.
> The proper solution would be to provide the name size, as read from the CPIO
> stream, and pass it to the entry.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
