[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354105&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354105
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:33
Start Date: 05/Dec/19 07:33
Worklog Time Spent: 10m
Work Description: coveralls commented on issue #84: COMPRESS-477 Add
support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#issuecomment-545874938
[](https://coveralls.io/builds/27429202)
Coverage increased (+0.2%) to 86.894% when pulling
**2b0775e362a0612b3b708d08dfe920152dbd9656 on PeterAlfreadLee:COMPRESS_477**
into **28049d2f89a76c5a4845c12151a483ca7773fb6f on apache:master**.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354105)
Time Spent: 9h 20m (was: 9h 10m)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-compress] coveralls edited a comment on issue #84: COMPRESS-477 Add support for extracting splitted zip files
coveralls edited a comment on issue #84: COMPRESS-477 Add support for extracting splitted zip files URL: https://github.com/apache/commons-compress/pull/84#issuecomment-545874938 [](https://coveralls.io/builds/27429202) Coverage increased (+0.2%) to 86.894% when pulling **2b0775e362a0612b3b708d08dfe920152dbd9656 on PeterAlfreadLee:COMPRESS_477** into **28049d2f89a76c5a4845c12151a483ca7773fb6f on apache:master**. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354098&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354098
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:25
Start Date: 05/Dec/19 07:25
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143028
##
File path:
src/main/java/org/apache/commons/compress/utils/ZipSplitReadOnlySeekableByteChannel.java
##
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.commons.compress.utils;
+
+import org.apache.commons.compress.archivers.ArchiveStreamFactory;
+import org.apache.commons.compress.archivers.zip.ZipLong;
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.channels.SeekableByteChannel;
+import java.nio.file.Files;
+import java.nio.file.StandardOpenOption;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.regex.Pattern;
+
+public class ZipSplitReadOnlySeekableByteChannel extends
MultiReadOnlySeekableByteChannel {
+protected final int ZIP_SPLIT_SIGNATURE_LENGTH = 4;
+protected ByteBuffer zipSplitSignatureByteBuffer =
ByteBuffer.allocate(ZIP_SPLIT_SIGNATURE_LENGTH);
+
+/**
+ * Concatenates the given channels.
+ * the channels should be add in ascending order, e.g. z01, z02, ... z99,
zip
+ * please note that the .zip file is the last segment and should be added
as the last one in the channels
+ *
+ * The first 4 bytes of split zip signature will be taken into
consideration by Inflator,
+ * so we don't need to skip them
+ *
+ * @param channels the channels to concatenate
+ * @throws NullPointerException if channels is null
+ */
+public ZipSplitReadOnlySeekableByteChannel(List
channels) throws IOException {
+super(channels);
+
+// each split zip segment should begin with zip split signature
+validSplitSignature(channels);
+}
+
+/**
+ * the first 4 bytes of zip split segments should be the zip split
signature(0x08074B50)
+ *
+ * @param channels channels to be valided
+ * @throws IOException
+ */
+private void validSplitSignature(final List channels)
throws IOException {
+for(int i = 0;i < channels.size();i++) {
+SeekableByteChannel channel = channels.get(i);
+// the zip split file signature is always at the beginning of the
segment
+channel.position(0L);
+
+channel.read(zipSplitSignatureByteBuffer);
+final ZipLong signature = new
ZipLong(zipSplitSignatureByteBuffer.array());
+if(!signature.equals(ZipLong.DD_SIG)) {
+channel.position(0L);
+throw new IOException("No." + (i + 1) + " split zip file is
not begin with split zip file signature");
+}
+
+channel.position(0L);
+}
+}
+
+/**
+ * set the position based on the given disk number and relative offset
+ *
+ * @param diskNumber the disk number of split zip files
+ * @param relativeOffset the offset in the split zip segment with given
disk number
+ * @return global position of all split zip files as if they are a single
channel
+ * @throws IOException
+ */
+public synchronized SeekableByteChannel position(long diskNumber, long
relativeOffset) throws IOException {
+long globalPosition = relativeOffset;
+for(int i = 0; i < diskNumber;i++) {
+globalPosition += channels.get(i).size();
+}
+
+return position(globalPosition);
+}
+
+/**
+ * Concatenates the given channels.
+ *
+ * @param channels the channels to concatenate, note that the LAST CHANNEL
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add
support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143177
##
File path:
src/main/java/org/apache/commons/compress/utils/ZipSplitReadOnlySeekableByteChannel.java
##
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.commons.compress.utils;
+
+import org.apache.commons.compress.archivers.ArchiveStreamFactory;
+import org.apache.commons.compress.archivers.zip.ZipLong;
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.channels.SeekableByteChannel;
+import java.nio.file.Files;
+import java.nio.file.StandardOpenOption;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.regex.Pattern;
+
+public class ZipSplitReadOnlySeekableByteChannel extends
MultiReadOnlySeekableByteChannel {
+protected final int ZIP_SPLIT_SIGNATURE_LENGTH = 4;
+protected ByteBuffer zipSplitSignatureByteBuffer =
ByteBuffer.allocate(ZIP_SPLIT_SIGNATURE_LENGTH);
+
+/**
+ * Concatenates the given channels.
+ * the channels should be add in ascending order, e.g. z01, z02, ... z99,
zip
+ * please note that the .zip file is the last segment and should be added
as the last one in the channels
+ *
+ * The first 4 bytes of split zip signature will be taken into
consideration by Inflator,
+ * so we don't need to skip them
+ *
+ * @param channels the channels to concatenate
+ * @throws NullPointerException if channels is null
+ */
+public ZipSplitReadOnlySeekableByteChannel(List
channels) throws IOException {
+super(channels);
+
+// each split zip segment should begin with zip split signature
+validSplitSignature(channels);
+}
+
+/**
+ * the first 4 bytes of zip split segments should be the zip split
signature(0x08074B50)
+ *
+ * @param channels channels to be valided
+ * @throws IOException
+ */
+private void validSplitSignature(final List channels)
throws IOException {
+for(int i = 0;i < channels.size();i++) {
+SeekableByteChannel channel = channels.get(i);
+// the zip split file signature is always at the beginning of the
segment
+channel.position(0L);
+
+channel.read(zipSplitSignatureByteBuffer);
+final ZipLong signature = new
ZipLong(zipSplitSignatureByteBuffer.array());
+if(!signature.equals(ZipLong.DD_SIG)) {
+channel.position(0L);
+throw new IOException("No." + (i + 1) + " split zip file is
not begin with split zip file signature");
+}
+
+channel.position(0L);
+}
+}
+
+/**
+ * set the position based on the given disk number and relative offset
+ *
+ * @param diskNumber the disk number of split zip files
+ * @param relativeOffset the offset in the split zip segment with given
disk number
+ * @return global position of all split zip files as if they are a single
channel
+ * @throws IOException
+ */
+public synchronized SeekableByteChannel position(long diskNumber, long
relativeOffset) throws IOException {
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354097&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354097
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:25
Start Date: 05/Dec/19 07:25
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143011
##
File path:
src/main/java/org/apache/commons/compress/utils/ZipSplitReadOnlySeekableByteChannel.java
##
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.commons.compress.utils;
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354097)
Time Spent: 8.5h (was: 8h 20m)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 8.5h
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add
support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143139
##
File path:
src/main/java/org/apache/commons/compress/utils/MultiReadOnlySeekableByteChannel.java
##
@@ -44,9 +44,9 @@
*/
public class MultiReadOnlySeekableByteChannel implements SeekableByteChannel {
-private final List channels;
-private long globalPosition;
-private int currentChannelIdx;
+protected final List channels;
Review comment:
This could be private now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354099&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354099
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:25
Start Date: 05/Dec/19 07:25
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143139
##
File path:
src/main/java/org/apache/commons/compress/utils/MultiReadOnlySeekableByteChannel.java
##
@@ -44,9 +44,9 @@
*/
public class MultiReadOnlySeekableByteChannel implements SeekableByteChannel {
-private final List channels;
-private long globalPosition;
-private int currentChannelIdx;
+protected final List channels;
Review comment:
This could be private now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354099)
Time Spent: 8h 50m (was: 8h 40m)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 8h 50m
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143221 ## File path: src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java ## @@ -143,6 +144,11 @@ */ private volatile boolean closed = true; +/** + * Whether the zip archive is a splite zip archive + */ +private boolean isSplitZipArchive = false; Review comment: Agree. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354101&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354101
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:25
Start Date: 05/Dec/19 07:25
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143221
##
File path: src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java
##
@@ -143,6 +144,11 @@
*/
private volatile boolean closed = true;
+/**
+ * Whether the zip archive is a splite zip archive
+ */
+private boolean isSplitZipArchive = false;
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354101)
Time Spent: 9h 10m (was: 9h)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 9h 10m
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354100&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354100
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:25
Start Date: 05/Dec/19 07:25
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143177
##
File path:
src/main/java/org/apache/commons/compress/utils/ZipSplitReadOnlySeekableByteChannel.java
##
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.commons.compress.utils;
+
+import org.apache.commons.compress.archivers.ArchiveStreamFactory;
+import org.apache.commons.compress.archivers.zip.ZipLong;
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.channels.SeekableByteChannel;
+import java.nio.file.Files;
+import java.nio.file.StandardOpenOption;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.regex.Pattern;
+
+public class ZipSplitReadOnlySeekableByteChannel extends
MultiReadOnlySeekableByteChannel {
+protected final int ZIP_SPLIT_SIGNATURE_LENGTH = 4;
+protected ByteBuffer zipSplitSignatureByteBuffer =
ByteBuffer.allocate(ZIP_SPLIT_SIGNATURE_LENGTH);
+
+/**
+ * Concatenates the given channels.
+ * the channels should be add in ascending order, e.g. z01, z02, ... z99,
zip
+ * please note that the .zip file is the last segment and should be added
as the last one in the channels
+ *
+ * The first 4 bytes of split zip signature will be taken into
consideration by Inflator,
+ * so we don't need to skip them
+ *
+ * @param channels the channels to concatenate
+ * @throws NullPointerException if channels is null
+ */
+public ZipSplitReadOnlySeekableByteChannel(List
channels) throws IOException {
+super(channels);
+
+// each split zip segment should begin with zip split signature
+validSplitSignature(channels);
+}
+
+/**
+ * the first 4 bytes of zip split segments should be the zip split
signature(0x08074B50)
+ *
+ * @param channels channels to be valided
+ * @throws IOException
+ */
+private void validSplitSignature(final List channels)
throws IOException {
+for(int i = 0;i < channels.size();i++) {
+SeekableByteChannel channel = channels.get(i);
+// the zip split file signature is always at the beginning of the
segment
+channel.position(0L);
+
+channel.read(zipSplitSignatureByteBuffer);
+final ZipLong signature = new
ZipLong(zipSplitSignatureByteBuffer.array());
+if(!signature.equals(ZipLong.DD_SIG)) {
+channel.position(0L);
+throw new IOException("No." + (i + 1) + " split zip file is
not begin with split zip file signature");
+}
+
+channel.position(0L);
+}
+}
+
+/**
+ * set the position based on the given disk number and relative offset
+ *
+ * @param diskNumber the disk number of split zip files
+ * @param relativeOffset the offset in the split zip segment with given
disk number
+ * @return global position of all split zip files as if they are a single
channel
+ * @throws IOException
+ */
+public synchronized SeekableByteChannel position(long diskNumber, long
relativeOffset) throws IOException {
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add
support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143028
##
File path:
src/main/java/org/apache/commons/compress/utils/ZipSplitReadOnlySeekableByteChannel.java
##
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.commons.compress.utils;
+
+import org.apache.commons.compress.archivers.ArchiveStreamFactory;
+import org.apache.commons.compress.archivers.zip.ZipLong;
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.channels.SeekableByteChannel;
+import java.nio.file.Files;
+import java.nio.file.StandardOpenOption;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.regex.Pattern;
+
+public class ZipSplitReadOnlySeekableByteChannel extends
MultiReadOnlySeekableByteChannel {
+protected final int ZIP_SPLIT_SIGNATURE_LENGTH = 4;
+protected ByteBuffer zipSplitSignatureByteBuffer =
ByteBuffer.allocate(ZIP_SPLIT_SIGNATURE_LENGTH);
+
+/**
+ * Concatenates the given channels.
+ * the channels should be add in ascending order, e.g. z01, z02, ... z99,
zip
+ * please note that the .zip file is the last segment and should be added
as the last one in the channels
+ *
+ * The first 4 bytes of split zip signature will be taken into
consideration by Inflator,
+ * so we don't need to skip them
+ *
+ * @param channels the channels to concatenate
+ * @throws NullPointerException if channels is null
+ */
+public ZipSplitReadOnlySeekableByteChannel(List
channels) throws IOException {
+super(channels);
+
+// each split zip segment should begin with zip split signature
+validSplitSignature(channels);
+}
+
+/**
+ * the first 4 bytes of zip split segments should be the zip split
signature(0x08074B50)
+ *
+ * @param channels channels to be valided
+ * @throws IOException
+ */
+private void validSplitSignature(final List channels)
throws IOException {
+for(int i = 0;i < channels.size();i++) {
+SeekableByteChannel channel = channels.get(i);
+// the zip split file signature is always at the beginning of the
segment
+channel.position(0L);
+
+channel.read(zipSplitSignatureByteBuffer);
+final ZipLong signature = new
ZipLong(zipSplitSignatureByteBuffer.array());
+if(!signature.equals(ZipLong.DD_SIG)) {
+channel.position(0L);
+throw new IOException("No." + (i + 1) + " split zip file is
not begin with split zip file signature");
+}
+
+channel.position(0L);
+}
+}
+
+/**
+ * set the position based on the given disk number and relative offset
+ *
+ * @param diskNumber the disk number of split zip files
+ * @param relativeOffset the offset in the split zip segment with given
disk number
+ * @return global position of all split zip files as if they are a single
channel
+ * @throws IOException
+ */
+public synchronized SeekableByteChannel position(long diskNumber, long
relativeOffset) throws IOException {
+long globalPosition = relativeOffset;
+for(int i = 0; i < diskNumber;i++) {
+globalPosition += channels.get(i).size();
+}
+
+return position(globalPosition);
+}
+
+/**
+ * Concatenates the given channels.
+ *
+ * @param channels the channels to concatenate, note that the LAST CHANNEL
of channels should be the LAST SEGMENT(.zip)
+ * and theses channels should be added in correct order
(e.g. .z01, .z02... .z99, .zip)
+ * @throws NullPointerException if channels is null
+ * @return SeekableByteChannel that concatenates all provided channels
+ */
+public static SeekableByteChannel
forSeekableByteChannels(SeekableByteChannel... channels) throws IOException {
+
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354096&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354096
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:24
Start Date: 05/Dec/19 07:24
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143001
##
File path:
src/main/java/org/apache/commons/compress/utils/ZipSplitSegmentComparator.java
##
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.commons.compress.utils;
+
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.File;
+import java.util.Comparator;
+
+public class ZipSplitSegmentComparator implements Comparator {
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354096)
Time Spent: 8h 20m (was: 8h 10m)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 8h 20m
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add
support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143001
##
File path:
src/main/java/org/apache/commons/compress/utils/ZipSplitSegmentComparator.java
##
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.commons.compress.utils;
+
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.File;
+import java.util.Comparator;
+
+public class ZipSplitSegmentComparator implements Comparator {
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354095&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354095
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:24
Start Date: 05/Dec/19 07:24
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354142987
##
File path:
src/main/java/org/apache/commons/compress/utils/ZipSplitReadOnlySeekableByteChannel.java
##
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.commons.compress.utils;
+
+import org.apache.commons.compress.archivers.ArchiveStreamFactory;
+import org.apache.commons.compress.archivers.zip.ZipLong;
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.channels.SeekableByteChannel;
+import java.nio.file.Files;
+import java.nio.file.StandardOpenOption;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.regex.Pattern;
+
+public class ZipSplitReadOnlySeekableByteChannel extends
MultiReadOnlySeekableByteChannel {
+protected final int ZIP_SPLIT_SIGNATURE_LENGTH = 4;
+protected ByteBuffer zipSplitSignatureByteBuffer =
ByteBuffer.allocate(ZIP_SPLIT_SIGNATURE_LENGTH);
+
+/**
+ * Concatenates the given channels.
+ * the channels should be add in ascending order, e.g. z01, z02, ... z99,
zip
+ * please note that the .zip file is the last segment and should be added
as the last one in the channels
+ *
+ * The first 4 bytes of split zip signature will be taken into
consideration by Inflator,
+ * so we don't need to skip them
+ *
+ * @param channels the channels to concatenate
+ * @throws NullPointerException if channels is null
+ */
+public ZipSplitReadOnlySeekableByteChannel(List
channels) throws IOException {
+super(channels);
+
+// each split zip segment should begin with zip split signature
+validSplitSignature(channels);
+}
+
+/**
+ * the first 4 bytes of zip split segments should be the zip split
signature(0x08074B50)
+ *
+ * @param channels channels to be valided
+ * @throws IOException
+ */
+private void validSplitSignature(final List channels)
throws IOException {
+for(int i = 0;i < channels.size();i++) {
+SeekableByteChannel channel = channels.get(i);
+// the zip split file signature is always at the beginning of the
segment
+channel.position(0L);
+
+channel.read(zipSplitSignatureByteBuffer);
+final ZipLong signature = new
ZipLong(zipSplitSignatureByteBuffer.array());
+if(!signature.equals(ZipLong.DD_SIG)) {
+channel.position(0L);
+throw new IOException("No." + (i + 1) + " split zip file is
not begin with split zip file signature");
+}
+
+channel.position(0L);
+}
+}
+
+/**
+ * set the position based on the given disk number and relative offset
+ *
+ * @param diskNumber the disk number of split zip files
+ * @param relativeOffset the offset in the split zip segment with given
disk number
+ * @return global position of all split zip files as if they are a single
channel
+ * @throws IOException
+ */
+public synchronized SeekableByteChannel position(long diskNumber, long
relativeOffset) throws IOException {
+long globalPosition = relativeOffset;
+for(int i = 0; i < diskNumber;i++) {
+globalPosition += channels.get(i).size();
+}
+
+return position(globalPosition);
+}
+
+/**
+ * Concatenates the given channels.
+ *
+ * @param channels the channels to concatenate, note that the LAST CHANNEL
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files URL: https://github.com/apache/commons-compress/pull/84#discussion_r354143011 ## File path: src/main/java/org/apache/commons/compress/utils/ZipSplitReadOnlySeekableByteChannel.java ## @@ -0,0 +1,214 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.commons.compress.utils; Review comment: Agree. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add
support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354142987
##
File path:
src/main/java/org/apache/commons/compress/utils/ZipSplitReadOnlySeekableByteChannel.java
##
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.commons.compress.utils;
+
+import org.apache.commons.compress.archivers.ArchiveStreamFactory;
+import org.apache.commons.compress.archivers.zip.ZipLong;
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.channels.SeekableByteChannel;
+import java.nio.file.Files;
+import java.nio.file.StandardOpenOption;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.regex.Pattern;
+
+public class ZipSplitReadOnlySeekableByteChannel extends
MultiReadOnlySeekableByteChannel {
+protected final int ZIP_SPLIT_SIGNATURE_LENGTH = 4;
+protected ByteBuffer zipSplitSignatureByteBuffer =
ByteBuffer.allocate(ZIP_SPLIT_SIGNATURE_LENGTH);
+
+/**
+ * Concatenates the given channels.
+ * the channels should be add in ascending order, e.g. z01, z02, ... z99,
zip
+ * please note that the .zip file is the last segment and should be added
as the last one in the channels
+ *
+ * The first 4 bytes of split zip signature will be taken into
consideration by Inflator,
+ * so we don't need to skip them
+ *
+ * @param channels the channels to concatenate
+ * @throws NullPointerException if channels is null
+ */
+public ZipSplitReadOnlySeekableByteChannel(List
channels) throws IOException {
+super(channels);
+
+// each split zip segment should begin with zip split signature
+validSplitSignature(channels);
+}
+
+/**
+ * the first 4 bytes of zip split segments should be the zip split
signature(0x08074B50)
+ *
+ * @param channels channels to be valided
+ * @throws IOException
+ */
+private void validSplitSignature(final List channels)
throws IOException {
+for(int i = 0;i < channels.size();i++) {
+SeekableByteChannel channel = channels.get(i);
+// the zip split file signature is always at the beginning of the
segment
+channel.position(0L);
+
+channel.read(zipSplitSignatureByteBuffer);
+final ZipLong signature = new
ZipLong(zipSplitSignatureByteBuffer.array());
+if(!signature.equals(ZipLong.DD_SIG)) {
+channel.position(0L);
+throw new IOException("No." + (i + 1) + " split zip file is
not begin with split zip file signature");
+}
+
+channel.position(0L);
+}
+}
+
+/**
+ * set the position based on the given disk number and relative offset
+ *
+ * @param diskNumber the disk number of split zip files
+ * @param relativeOffset the offset in the split zip segment with given
disk number
+ * @return global position of all split zip files as if they are a single
channel
+ * @throws IOException
+ */
+public synchronized SeekableByteChannel position(long diskNumber, long
relativeOffset) throws IOException {
+long globalPosition = relativeOffset;
+for(int i = 0; i < diskNumber;i++) {
+globalPosition += channels.get(i).size();
+}
+
+return position(globalPosition);
+}
+
+/**
+ * Concatenates the given channels.
+ *
+ * @param channels the channels to concatenate, note that the LAST CHANNEL
of channels should be the LAST SEGMENT(.zip)
+ * and theses channels should be added in correct order
(e.g. .z01, .z02... .z99, .zip)
+ * @throws NullPointerException if channels is null
+ * @return SeekableByteChannel that concatenates all provided channels
+ */
+public static SeekableByteChannel
forSeekableByteChannels(SeekableByteChannel... channels) throws IOException {
+
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354093&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354093
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:23
Start Date: 05/Dec/19 07:23
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #86:
COMPRESS-477 building a split zip
URL: https://github.com/apache/commons-compress/pull/86#discussion_r354142521
##
File path:
src/main/java/org/apache/commons/compress/archivers/zip/ZipSplitOutputStream.java
##
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.commons.compress.archivers.zip;
+
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+
+public class ZipSplitOutputStream extends OutputStream {
+private OutputStream outputStream;
+private File zipFile;
+private long splitSize;
+private int currentSplitSegmentIndex = 0;
+private long currentSplitSegmentBytesWritten = 0;
+private boolean finished = false;
+
+/**
+ * 8.5.1 Capacities for split archives are as follows:
+ *
+ * Maximum number of segments = 4,294,967,295 - 1
+ * Maximum .ZIP segment size = 4,294,967,295 bytes (refer to section 8.5.6)
+ * Minimum segment size = 64K
+ * Maximum PKSFX segment size = 2,147,483,647 bytes
+ */
+private final long ZIP_SEGMENT_MIN_SIZE = 64 * 1024L;
+private final long ZIP_SEGMENT_MAX_SIZE = 4294967295L;
+
+/**
+ * Create a split zip. If the zip file is smaller than the split size,
+ * then there will only be one split zip, and its suffix is .zip,
+ * otherwise the split segments should be like .z01, .z02, ... .z(N-1),
.zip
+ *
+ * @param zipFile the zip file to write to
+ * @param splitSize the split size
+ */
+public ZipSplitOutputStream(final File zipFile, final long splitSize)
throws IllegalArgumentException, IOException {
+if (splitSize < ZIP_SEGMENT_MIN_SIZE || splitSize >
ZIP_SEGMENT_MAX_SIZE) {
+throw new IllegalArgumentException("zip split segment size should
between 64K and 4,294,967,295");
+}
+
+this.zipFile = zipFile;
+this.splitSize = splitSize;
+
+this.outputStream = new FileOutputStream(zipFile);
+// write the zip split signature 0x08074B50 to the zip file
+writeZipSplitSignature();
+}
+
+/**
+ * Some data can not be written to different split segments, for example:
+ *
+ * 4.4.1.5 The end of central directory record and the Zip64 end
+ * of central directory locator record MUST reside on the same
+ * disk when splitting or spanning an archive.
+ *
+ * @param unsplittableContentSize
+ * @throws IllegalArgumentException
+ * @throws IOException
+ */
+public void prepareToWriteUnsplittableContent(long
unsplittableContentSize) throws IllegalArgumentException, IOException {
+if (unsplittableContentSize > this.splitSize) {
+throw new IllegalArgumentException("The unsplittable content size
is bigger than the split segment size");
+}
+
+long bytesRemainingInThisSegment = this.splitSize -
this.currentSplitSegmentBytesWritten;
+if (bytesRemainingInThisSegment < unsplittableContentSize) {
+openNewSplitSegment();
+}
+}
+
+@Override
+public void write(int i) throws IOException {
+byte[] b = ByteBuffer.allocate(4).putInt(i).array();
Review comment:
Agree. This should no doubt be a single byte. Sorry for my careless.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354093)
Time Spent: 8h (was:
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #86: COMPRESS-477 building a split zip
PeterAlfreadLee commented on a change in pull request #86: COMPRESS-477
building a split zip
URL: https://github.com/apache/commons-compress/pull/86#discussion_r354142521
##
File path:
src/main/java/org/apache/commons/compress/archivers/zip/ZipSplitOutputStream.java
##
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.commons.compress.archivers.zip;
+
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+
+public class ZipSplitOutputStream extends OutputStream {
+private OutputStream outputStream;
+private File zipFile;
+private long splitSize;
+private int currentSplitSegmentIndex = 0;
+private long currentSplitSegmentBytesWritten = 0;
+private boolean finished = false;
+
+/**
+ * 8.5.1 Capacities for split archives are as follows:
+ *
+ * Maximum number of segments = 4,294,967,295 - 1
+ * Maximum .ZIP segment size = 4,294,967,295 bytes (refer to section 8.5.6)
+ * Minimum segment size = 64K
+ * Maximum PKSFX segment size = 2,147,483,647 bytes
+ */
+private final long ZIP_SEGMENT_MIN_SIZE = 64 * 1024L;
+private final long ZIP_SEGMENT_MAX_SIZE = 4294967295L;
+
+/**
+ * Create a split zip. If the zip file is smaller than the split size,
+ * then there will only be one split zip, and its suffix is .zip,
+ * otherwise the split segments should be like .z01, .z02, ... .z(N-1),
.zip
+ *
+ * @param zipFile the zip file to write to
+ * @param splitSize the split size
+ */
+public ZipSplitOutputStream(final File zipFile, final long splitSize)
throws IllegalArgumentException, IOException {
+if (splitSize < ZIP_SEGMENT_MIN_SIZE || splitSize >
ZIP_SEGMENT_MAX_SIZE) {
+throw new IllegalArgumentException("zip split segment size should
between 64K and 4,294,967,295");
+}
+
+this.zipFile = zipFile;
+this.splitSize = splitSize;
+
+this.outputStream = new FileOutputStream(zipFile);
+// write the zip split signature 0x08074B50 to the zip file
+writeZipSplitSignature();
+}
+
+/**
+ * Some data can not be written to different split segments, for example:
+ *
+ * 4.4.1.5 The end of central directory record and the Zip64 end
+ * of central directory locator record MUST reside on the same
+ * disk when splitting or spanning an archive.
+ *
+ * @param unsplittableContentSize
+ * @throws IllegalArgumentException
+ * @throws IOException
+ */
+public void prepareToWriteUnsplittableContent(long
unsplittableContentSize) throws IllegalArgumentException, IOException {
+if (unsplittableContentSize > this.splitSize) {
+throw new IllegalArgumentException("The unsplittable content size
is bigger than the split segment size");
+}
+
+long bytesRemainingInThisSegment = this.splitSize -
this.currentSplitSegmentBytesWritten;
+if (bytesRemainingInThisSegment < unsplittableContentSize) {
+openNewSplitSegment();
+}
+}
+
+@Override
+public void write(int i) throws IOException {
+byte[] b = ByteBuffer.allocate(4).putInt(i).array();
Review comment:
Agree. This should no doubt be a single byte. Sorry for my careless.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #86: COMPRESS-477 building a split zip
PeterAlfreadLee commented on a change in pull request #86: COMPRESS-477
building a split zip
URL: https://github.com/apache/commons-compress/pull/86#discussion_r354142088
##
File path:
src/main/java/org/apache/commons/compress/archivers/zip/ZipSplitOutputStream.java
##
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.commons.compress.archivers.zip;
+
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+
+public class ZipSplitOutputStream extends OutputStream {
+private OutputStream outputStream;
+private File zipFile;
+private long splitSize;
Review comment:
Agree
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354090&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354090
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:21
Start Date: 05/Dec/19 07:21
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #86:
COMPRESS-477 building a split zip
URL: https://github.com/apache/commons-compress/pull/86#discussion_r354142056
##
File path:
src/main/java/org/apache/commons/compress/compressors/FileNameUtil.java
##
@@ -193,4 +193,19 @@ public String getCompressedFilename(final String
fileName) {
return fileName + defaultExtension;
}
+public static String getBaseName(String filename) {
+if (filename == null) {
+return null;
+}
+
+int lastSeparatorPosition = Math.max(filename.lastIndexOf('/'),
filename.lastIndexOf('\\'));
+String name = filename.substring(lastSeparatorPosition + 1);
Review comment:
Agree
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354090)
Time Spent: 7.5h (was: 7h 20m)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #86: COMPRESS-477 building a split zip
PeterAlfreadLee commented on a change in pull request #86: COMPRESS-477
building a split zip
URL: https://github.com/apache/commons-compress/pull/86#discussion_r354142056
##
File path:
src/main/java/org/apache/commons/compress/compressors/FileNameUtil.java
##
@@ -193,4 +193,19 @@ public String getCompressedFilename(final String
fileName) {
return fileName + defaultExtension;
}
+public static String getBaseName(String filename) {
+if (filename == null) {
+return null;
+}
+
+int lastSeparatorPosition = Math.max(filename.lastIndexOf('/'),
filename.lastIndexOf('\\'));
+String name = filename.substring(lastSeparatorPosition + 1);
Review comment:
Agree
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #86: COMPRESS-477 building a split zip
PeterAlfreadLee commented on a change in pull request #86: COMPRESS-477
building a split zip
URL: https://github.com/apache/commons-compress/pull/86#discussion_r354142071
##
File path:
src/main/java/org/apache/commons/compress/archivers/zip/ZipSplitOutputStream.java
##
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.commons.compress.archivers.zip;
+
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+
+public class ZipSplitOutputStream extends OutputStream {
+private OutputStream outputStream;
+private File zipFile;
+private long splitSize;
+private int currentSplitSegmentIndex = 0;
+private long currentSplitSegmentBytesWritten = 0;
+private boolean finished = false;
+
+/**
+ * 8.5.1 Capacities for split archives are as follows:
+ *
+ * Maximum number of segments = 4,294,967,295 - 1
+ * Maximum .ZIP segment size = 4,294,967,295 bytes (refer to section 8.5.6)
+ * Minimum segment size = 64K
+ * Maximum PKSFX segment size = 2,147,483,647 bytes
+ */
+private final long ZIP_SEGMENT_MIN_SIZE = 64 * 1024L;
+private final long ZIP_SEGMENT_MAX_SIZE = 4294967295L;
+
+/**
+ * Create a split zip. If the zip file is smaller than the split size,
+ * then there will only be one split zip, and its suffix is .zip,
+ * otherwise the split segments should be like .z01, .z02, ... .z(N-1),
.zip
+ *
+ * @param zipFile the zip file to write to
+ * @param splitSize the split size
+ */
+public ZipSplitOutputStream(final File zipFile, final long splitSize)
throws IllegalArgumentException, IOException {
+if (splitSize < ZIP_SEGMENT_MIN_SIZE || splitSize >
ZIP_SEGMENT_MAX_SIZE) {
+throw new IllegalArgumentException("zip split segment size should
between 64K and 4,294,967,295");
+}
+
+this.zipFile = zipFile;
+this.splitSize = splitSize;
+
+this.outputStream = new FileOutputStream(zipFile);
+// write the zip split signature 0x08074B50 to the zip file
+writeZipSplitSignature();
+}
+
+/**
+ * Some data can not be written to different split segments, for example:
+ *
+ * 4.4.1.5 The end of central directory record and the Zip64 end
+ * of central directory locator record MUST reside on the same
+ * disk when splitting or spanning an archive.
+ *
+ * @param unsplittableContentSize
+ * @throws IllegalArgumentException
+ * @throws IOException
+ */
+public void prepareToWriteUnsplittableContent(long
unsplittableContentSize) throws IllegalArgumentException, IOException {
+if (unsplittableContentSize > this.splitSize) {
+throw new IllegalArgumentException("The unsplittable content size
is bigger than the split segment size");
+}
+
+long bytesRemainingInThisSegment = this.splitSize -
this.currentSplitSegmentBytesWritten;
+if (bytesRemainingInThisSegment < unsplittableContentSize) {
+openNewSplitSegment();
+}
+}
+
+@Override
+public void write(int i) throws IOException {
+byte[] b = ByteBuffer.allocate(4).putInt(i).array();
+write(b);
+}
+
+@Override
+public void write(byte[] b) throws IOException {
+write(b, 0, b.length);
+}
+
+/**
+ * Write the data to zip split segments, if the remaining space of current
split segment
+ * is not enough, then a new split segment should be created
+ *
+ * @param b data to write
+ * @param off offset of the start of data in param b
+ * @param len the length of data to write
+ * @throws IOException
+ */
+@Override
+public void write(byte[] b, int off, int len) throws IOException {
+if (len <= 0) {
+return;
+}
+
+if (currentSplitSegmentBytesWritten >= splitSize) {
+openNewSplitSegment();
+write(b, off, len);
+} else if (currentSplitSegmentBytesWritten + len > splitSize) {
+int bytesToWriteForThisSegment = (int) splitSize - (int)
currentSplitSegmentBytesW
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354091&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354091
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:21
Start Date: 05/Dec/19 07:21
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #86:
COMPRESS-477 building a split zip
URL: https://github.com/apache/commons-compress/pull/86#discussion_r354142071
##
File path:
src/main/java/org/apache/commons/compress/archivers/zip/ZipSplitOutputStream.java
##
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.commons.compress.archivers.zip;
+
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+
+public class ZipSplitOutputStream extends OutputStream {
+private OutputStream outputStream;
+private File zipFile;
+private long splitSize;
+private int currentSplitSegmentIndex = 0;
+private long currentSplitSegmentBytesWritten = 0;
+private boolean finished = false;
+
+/**
+ * 8.5.1 Capacities for split archives are as follows:
+ *
+ * Maximum number of segments = 4,294,967,295 - 1
+ * Maximum .ZIP segment size = 4,294,967,295 bytes (refer to section 8.5.6)
+ * Minimum segment size = 64K
+ * Maximum PKSFX segment size = 2,147,483,647 bytes
+ */
+private final long ZIP_SEGMENT_MIN_SIZE = 64 * 1024L;
+private final long ZIP_SEGMENT_MAX_SIZE = 4294967295L;
+
+/**
+ * Create a split zip. If the zip file is smaller than the split size,
+ * then there will only be one split zip, and its suffix is .zip,
+ * otherwise the split segments should be like .z01, .z02, ... .z(N-1),
.zip
+ *
+ * @param zipFile the zip file to write to
+ * @param splitSize the split size
+ */
+public ZipSplitOutputStream(final File zipFile, final long splitSize)
throws IllegalArgumentException, IOException {
+if (splitSize < ZIP_SEGMENT_MIN_SIZE || splitSize >
ZIP_SEGMENT_MAX_SIZE) {
+throw new IllegalArgumentException("zip split segment size should
between 64K and 4,294,967,295");
+}
+
+this.zipFile = zipFile;
+this.splitSize = splitSize;
+
+this.outputStream = new FileOutputStream(zipFile);
+// write the zip split signature 0x08074B50 to the zip file
+writeZipSplitSignature();
+}
+
+/**
+ * Some data can not be written to different split segments, for example:
+ *
+ * 4.4.1.5 The end of central directory record and the Zip64 end
+ * of central directory locator record MUST reside on the same
+ * disk when splitting or spanning an archive.
+ *
+ * @param unsplittableContentSize
+ * @throws IllegalArgumentException
+ * @throws IOException
+ */
+public void prepareToWriteUnsplittableContent(long
unsplittableContentSize) throws IllegalArgumentException, IOException {
+if (unsplittableContentSize > this.splitSize) {
+throw new IllegalArgumentException("The unsplittable content size
is bigger than the split segment size");
+}
+
+long bytesRemainingInThisSegment = this.splitSize -
this.currentSplitSegmentBytesWritten;
+if (bytesRemainingInThisSegment < unsplittableContentSize) {
+openNewSplitSegment();
+}
+}
+
+@Override
+public void write(int i) throws IOException {
+byte[] b = ByteBuffer.allocate(4).putInt(i).array();
+write(b);
+}
+
+@Override
+public void write(byte[] b) throws IOException {
+write(b, 0, b.length);
+}
+
+/**
+ * Write the data to zip split segments, if the remaining space of current
split segment
+ * is not enough, then a new split segment should be created
+ *
+ * @param b data to write
+ * @param off offset of the start of data in param b
+ * @param len the length of data to write
+ * @throws IOException
+ */
+@Override
+pu
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354092&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354092
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 07:21
Start Date: 05/Dec/19 07:21
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #86:
COMPRESS-477 building a split zip
URL: https://github.com/apache/commons-compress/pull/86#discussion_r354142088
##
File path:
src/main/java/org/apache/commons/compress/archivers/zip/ZipSplitOutputStream.java
##
@@ -0,0 +1,236 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+package org.apache.commons.compress.archivers.zip;
+
+import org.apache.commons.compress.compressors.FileNameUtil;
+
+import java.io.*;
+import java.nio.ByteBuffer;
+
+public class ZipSplitOutputStream extends OutputStream {
+private OutputStream outputStream;
+private File zipFile;
+private long splitSize;
Review comment:
Agree
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354092)
Time Spent: 7h 50m (was: 7h 40m)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354030&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354030
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 04:02
Start Date: 05/Dec/19 04:02
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354101926
##
File path:
src/main/java/org/apache/commons/compress/archivers/zip/ZipArchiveInputStream.java
##
@@ -367,13 +367,9 @@ public ZipArchiveEntry getNextZipEntry() throws
IOException {
private void readFirstLocalFileHeader(final byte[] lfh) throws IOException
{
readFully(lfh);
final ZipLong sig = new ZipLong(lfh);
-if (sig.equals(ZipLong.DD_SIG)) {
Review comment:
I don't fully understand this. I'm a little confused about this.
My understanding about this review is :
We should add a new flag like `skipSplitZipSig` in the constructor of
`ZipArchiveInputStream`. Then we will only skip the `ZipLong.DD_SIG` signature
when the `skipSplitZipSig` is set.
Am I right?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354030)
Time Spent: 7h 20m (was: 7h 10m)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add
support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354101926
##
File path:
src/main/java/org/apache/commons/compress/archivers/zip/ZipArchiveInputStream.java
##
@@ -367,13 +367,9 @@ public ZipArchiveEntry getNextZipEntry() throws
IOException {
private void readFirstLocalFileHeader(final byte[] lfh) throws IOException
{
readFully(lfh);
final ZipLong sig = new ZipLong(lfh);
-if (sig.equals(ZipLong.DD_SIG)) {
Review comment:
I don't fully understand this. I'm a little confused about this.
My understanding about this review is :
We should add a new flag like `skipSplitZipSig` in the constructor of
`ZipArchiveInputStream`. Then we will only skip the `ZipLong.DD_SIG` signature
when the `skipSplitZipSig` is set.
Am I right?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354022&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354022
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 03:28
Start Date: 05/Dec/19 03:28
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354095888
##
File path: src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java
##
@@ -143,6 +144,11 @@
*/
private volatile boolean closed = true;
+/**
+ * Whether the zip archive is a splite zip archive
+ */
+private boolean isSplitZipArchive = false;
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354022)
Time Spent: 6h 40m (was: 6.5h)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354024&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354024
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 03:28
Start Date: 05/Dec/19 03:28
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354095921
##
File path:
src/main/java/org/apache/commons/compress/utils/MultiReadOnlySeekableByteChannel.java
##
@@ -44,9 +44,9 @@
*/
public class MultiReadOnlySeekableByteChannel implements SeekableByteChannel {
-private final List channels;
-private long globalPosition;
-private int currentChannelIdx;
+protected final List channels;
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354024)
Time Spent: 6h 50m (was: 6h 40m)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354026&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354026
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 03:28
Start Date: 05/Dec/19 03:28
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354095921
##
File path:
src/main/java/org/apache/commons/compress/utils/MultiReadOnlySeekableByteChannel.java
##
@@ -44,9 +44,9 @@
*/
public class MultiReadOnlySeekableByteChannel implements SeekableByteChannel {
-private final List channels;
-private long globalPosition;
-private int currentChannelIdx;
+protected final List channels;
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354026)
Time Spent: 7h 10m (was: 7h)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (COMPRESS-477) Support for splitted zip files
[
https://issues.apache.org/jira/browse/COMPRESS-477?focusedWorklogId=354025&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354025
]
ASF GitHub Bot logged work on COMPRESS-477:
---
Author: ASF GitHub Bot
Created on: 05/Dec/19 03:28
Start Date: 05/Dec/19 03:28
Worklog Time Spent: 10m
Work Description: PeterAlfreadLee commented on pull request #84:
COMPRESS-477 Add support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354095888
##
File path: src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java
##
@@ -143,6 +144,11 @@
*/
private volatile boolean closed = true;
+/**
+ * Whether the zip archive is a splite zip archive
+ */
+private boolean isSplitZipArchive = false;
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 354025)
Time Spent: 7h (was: 6h 50m)
> Support for splitted zip files
> --
>
> Key: COMPRESS-477
> URL: https://issues.apache.org/jira/browse/COMPRESS-477
> Project: Commons Compress
> Issue Type: New Feature
> Components: Archivers
>Affects Versions: 1.18
>Reporter: Luís Filipe Nassif
>Priority: Major
> Labels: zip
> Fix For: 1.20
>
> Time Spent: 7h
> Remaining Estimate: 0h
>
> It would be very useful to support splitted zip files. I've read
> [https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT] and understood
> that simply concatenating the segments and removing the split signature
> 0x08074b50 from first segment would be sufficient, but it is not that simple
> because compress fails with exception below:
> {code}
> Caused by: java.util.zip.ZipException: archive's ZIP64 end of central
> directory locator is corrupt.
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory64(ZipFile.java:924)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.positionAtCentralDirectory(ZipFile.java:901)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.populateFromCentralDirectory(ZipFile.java:621)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:295)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:280)
> ~[commons-compress-1.18.jar:1.18]
> at
> org.apache.commons.compress.archivers.zip.ZipFile.(ZipFile.java:236)
> ~[commons-compress-1.18.jar:1.18]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files URL: https://github.com/apache/commons-compress/pull/84#discussion_r354095888 ## File path: src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java ## @@ -143,6 +144,11 @@ */ private volatile boolean closed = true; +/** + * Whether the zip archive is a splite zip archive + */ +private boolean isSplitZipArchive = false; Review comment: Agree. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files URL: https://github.com/apache/commons-compress/pull/84#discussion_r354095888 ## File path: src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java ## @@ -143,6 +144,11 @@ */ private volatile boolean closed = true; +/** + * Whether the zip archive is a splite zip archive + */ +private boolean isSplitZipArchive = false; Review comment: Agree. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add
support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354095921
##
File path:
src/main/java/org/apache/commons/compress/utils/MultiReadOnlySeekableByteChannel.java
##
@@ -44,9 +44,9 @@
*/
public class MultiReadOnlySeekableByteChannel implements SeekableByteChannel {
-private final List channels;
-private long globalPosition;
-private int currentChannelIdx;
+protected final List channels;
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [commons-compress] PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add support for extracting splitted zip files
PeterAlfreadLee commented on a change in pull request #84: COMPRESS-477 Add
support for extracting splitted zip files
URL: https://github.com/apache/commons-compress/pull/84#discussion_r354095921
##
File path:
src/main/java/org/apache/commons/compress/utils/MultiReadOnlySeekableByteChannel.java
##
@@ -44,9 +44,9 @@
*/
public class MultiReadOnlySeekableByteChannel implements SeekableByteChannel {
-private final List channels;
-private long globalPosition;
-private int currentChannelIdx;
+protected final List channels;
Review comment:
Agree.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [commons-collections] dota17 commented on a change in pull request #122: Improve MapUtils with the null checks, add Javadoc for the parameter indent and add JUnit for it.
dota17 commented on a change in pull request #122: Improve MapUtils with the
null checks, add Javadoc for the parameter indent and add JUnit for it.
URL:
https://github.com/apache/commons-collections/pull/122#discussion_r354082048
##
File path: src/main/java/org/apache/commons/collections4/MapUtils.java
##
@@ -1186,11 +1188,12 @@ public static void debugPrint(final PrintStream out,
final Object label, final M
*
* @param the key type
* @param the value type
- * @param map the map to invert, may not be null
+ * @param map the map to invert, must not be null
* @return a new HashMap containing the inverted data
* @throws NullPointerException if the map is null
*/
public static Map invertMap(final Map map) {
+Objects.requireNonNull(map, "The map must not be null");
Review comment:
Thanks for your advice. I have updated it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work logged] (COLLECTIONS-740) Missing @throws comment in SwitchTransformer.switchTransformer
[
https://issues.apache.org/jira/browse/COLLECTIONS-740?focusedWorklogId=353996&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-353996
]
ASF GitHub Bot logged work on COLLECTIONS-740:
--
Author: ASF GitHub Bot
Created on: 05/Dec/19 01:58
Start Date: 05/Dec/19 01:58
Worklog Time Spent: 10m
Work Description: dota17 commented on pull request #124:
[COLLECTIONS-740] Add missing @throws comment for
SwitchTransformer.switchTransformer.
URL:
https://github.com/apache/commons-collections/pull/124#discussion_r354077393
##
File path:
src/main/java/org/apache/commons/collections4/functors/SwitchTransformer.java
##
@@ -51,6 +51,7 @@
* @return the chained transformer
* @throws NullPointerException if array is null
* @throws NullPointerException if any element in the array is null
+ * @throws IllegalArgumentException if predicate and transformer arrays do
not have the same size
Review comment:
maybe it's better to be the same as TransformerUtils line 317
` @throws IllegalArgumentException if the arrays have different sizes`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 353996)
Remaining Estimate: 0h
Time Spent: 10m
> Missing @throws comment in SwitchTransformer.switchTransformer
> --
>
> Key: COLLECTIONS-740
> URL: https://issues.apache.org/jira/browse/COLLECTIONS-740
> Project: Commons Collections
> Issue Type: Improvement
> Components: Functor
>Reporter: Pengyu Nie
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> SwitchTransformer.switchTransformer could throw IllegalArgumentException if
> predicate and transformer arrays have different lengths, but that's not
> described in Javadoc. See the method below, the exception may happen in the
> first if statement:
> {code:java}
> /**
> * Factory method that performs validation and copies the parameter arrays.
> *
> * @param the input type
> * @param the output type
> * @param predicates array of predicates, cloned, no nulls
> * @param transformers matching array of transformers, cloned, no nulls
> * @param defaultTransformer the transformer to use if no match, null means
> return null
> * @return the chained transformer
> * @throws NullPointerException if array is null
> * @throws NullPointerException if any element in the array is null
> * @throws IllegalAccessException if predicate and transformer arrays do not
> have the same size
> */
> @SuppressWarnings("unchecked")
> public static Transformer switchTransformer(final Predicate super I>[] predicates,
> final Transformer[] transformers,
> final Transformer defaultTransformer) {
> FunctorUtils.validate(predicates);
> FunctorUtils.validate(transformers);
> if (predicates.length != transformers.length) {
> throw new IllegalArgumentException("The predicate and transformer
> arrays must be the same size");
> }
> if (predicates.length == 0) {
> return (Transformer) (defaultTransformer == null ?
> ConstantTransformer.nullTransformer() :
>
> defaultTransformer);
> }
> return new SwitchTransformer<>(predicates, transformers,
> defaultTransformer);
> }
> {code}
> I'll submit a PR to add the missing @throws documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-collections] dota17 commented on a change in pull request #124: [COLLECTIONS-740] Add missing @throws comment for SwitchTransformer.switchTransformer.
dota17 commented on a change in pull request #124: [COLLECTIONS-740] Add missing @throws comment for SwitchTransformer.switchTransformer. URL: https://github.com/apache/commons-collections/pull/124#discussion_r354077393 ## File path: src/main/java/org/apache/commons/collections4/functors/SwitchTransformer.java ## @@ -51,6 +51,7 @@ * @return the chained transformer * @throws NullPointerException if array is null * @throws NullPointerException if any element in the array is null + * @throws IllegalArgumentException if predicate and transformer arrays do not have the same size Review comment: maybe it's better to be the same as TransformerUtils line 317 ` @throws IllegalArgumentException if the arrays have different sizes` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (COLLECTIONS-739) Wrong @throws comments in DefaultedMap
[
https://issues.apache.org/jira/browse/COLLECTIONS-739?focusedWorklogId=353978&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-353978
]
ASF GitHub Bot logged work on COLLECTIONS-739:
--
Author: ASF GitHub Bot
Created on: 05/Dec/19 01:32
Start Date: 05/Dec/19 01:32
Worklog Time Spent: 10m
Work Description: dota17 commented on pull request #123:
[COLLECTIONS-739] Fix inconsistent @throws comments in DefaultedMap
URL:
https://github.com/apache/commons-collections/pull/123#discussion_r354071920
##
File path: src/main/java/org/apache/commons/collections4/map/DefaultedMap.java
##
@@ -105,7 +105,8 @@
* @param map the map to decorate, must not be null
* @param factory the factory to use to create entries, must not be null
* @return a new defaulting map
- * @throws NullPointerException if map or factory is null
+ * @throws NullPointerException if map is null
+ * @throws IllegalArgumentException if factory is null
Review comment:
This is indeed a judgment of NullPointerException. It is recommended to
modify IllegalArgumentException to NullPointerException, and to supplement the
test cases for it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 353978)
Time Spent: 20m (was: 10m)
> Wrong @throws comments in DefaultedMap
> --
>
> Key: COLLECTIONS-739
> URL: https://issues.apache.org/jira/browse/COLLECTIONS-739
> Project: Commons Collections
> Issue Type: Improvement
> Components: Map
>Reporter: Pengyu Nie
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> I found some @throws comments in DefaultedMap are inconsistent with the code
> or missing. For examples:
> 1. The comment for defaultedMap(Map, Factory) has wrong
> exception type:
> Line 108:
> {code:java}
> /** ... @throws NullPointerException if map or factory is null ... */
> {code}
> Line 112-114:
> {code:java}
> if (factory == null) {
> throw new IllegalArgumentException("Factory must not be null");
> }
> {code}
> 2. For defaultMap(Map, Transformer, this null
> checking does not have corresponding @throws comment:
> Line 135-137:
> {code:java}
> if (transformer == null) {
> throw new IllegalArgumentException("Transformer must not be null");
> }
> {code}
> I suggest adding:
> {code:java}
> @throws IllegalArgumentException if transformer is null.
> {code}
> I'll submit a PR for these suggested changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-collections] dota17 commented on a change in pull request #123: [COLLECTIONS-739] Fix inconsistent @throws comments in DefaultedMap
dota17 commented on a change in pull request #123: [COLLECTIONS-739] Fix inconsistent @throws comments in DefaultedMap URL: https://github.com/apache/commons-collections/pull/123#discussion_r354071920 ## File path: src/main/java/org/apache/commons/collections4/map/DefaultedMap.java ## @@ -105,7 +105,8 @@ * @param map the map to decorate, must not be null * @param factory the factory to use to create entries, must not be null * @return a new defaulting map - * @throws NullPointerException if map or factory is null + * @throws NullPointerException if map is null + * @throws IllegalArgumentException if factory is null Review comment: This is indeed a judgment of NullPointerException. It is recommended to modify IllegalArgumentException to NullPointerException, and to supplement the test cases for it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (COLLECTIONS-739) Wrong @throws comments in DefaultedMap
[
https://issues.apache.org/jira/browse/COLLECTIONS-739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988368#comment-16988368
]
Chen Guoping commented on COLLECTIONS-739:
--
Hi Pengyu Nie:
1. The comment for defaultedMap(Map, Factory) has wrong
exception type:
Line 108:
/** ... @throws NullPointerException if map or factory is null ... */
Line 112-114:
{code:java}
if (factory == null) {
throw new NullPointerException("Factory must not be null");
}{code}
2. For defaultMap(Map, Transformer, this null
checking does not have corresponding @throws comment:
Line 135-137:
{code:java}
if (transformer == null) {
throw new NullPointerException("Transformer must not be null");
}{code}
I suggest that updating IllegalArgumentException to NullPointerException .
Best wishes and regards,
Chen
> Wrong @throws comments in DefaultedMap
> --
>
> Key: COLLECTIONS-739
> URL: https://issues.apache.org/jira/browse/COLLECTIONS-739
> Project: Commons Collections
> Issue Type: Improvement
> Components: Map
>Reporter: Pengyu Nie
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I found some @throws comments in DefaultedMap are inconsistent with the code
> or missing. For examples:
> 1. The comment for defaultedMap(Map, Factory) has wrong
> exception type:
> Line 108:
> {code:java}
> /** ... @throws NullPointerException if map or factory is null ... */
> {code}
> Line 112-114:
> {code:java}
> if (factory == null) {
> throw new IllegalArgumentException("Factory must not be null");
> }
> {code}
> 2. For defaultMap(Map, Transformer, this null
> checking does not have corresponding @throws comment:
> Line 135-137:
> {code:java}
> if (transformer == null) {
> throw new IllegalArgumentException("Transformer must not be null");
> }
> {code}
> I suggest adding:
> {code:java}
> @throws IllegalArgumentException if transformer is null.
> {code}
> I'll submit a PR for these suggested changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (COLLECTIONS-740) Missing @throws comment in SwitchTransformer.switchTransformer
Pengyu Nie created COLLECTIONS-740:
--
Summary: Missing @throws comment in
SwitchTransformer.switchTransformer
Key: COLLECTIONS-740
URL: https://issues.apache.org/jira/browse/COLLECTIONS-740
Project: Commons Collections
Issue Type: Improvement
Components: Functor
Reporter: Pengyu Nie
SwitchTransformer.switchTransformer could throw IllegalArgumentException if
predicate and transformer arrays have different lengths, but that's not
described in Javadoc. See the method below, the exception may happen in the
first if statement:
{code:java}
/**
* Factory method that performs validation and copies the parameter arrays.
*
* @param the input type
* @param the output type
* @param predicates array of predicates, cloned, no nulls
* @param transformers matching array of transformers, cloned, no nulls
* @param defaultTransformer the transformer to use if no match, null means
return null
* @return the chained transformer
* @throws NullPointerException if array is null
* @throws NullPointerException if any element in the array is null
* @throws IllegalAccessException if predicate and transformer arrays do not
have the same size
*/
@SuppressWarnings("unchecked")
public static Transformer switchTransformer(final Predicate[] predicates,
final Transformer[] transformers,
final Transformer defaultTransformer) {
FunctorUtils.validate(predicates);
FunctorUtils.validate(transformers);
if (predicates.length != transformers.length) {
throw new IllegalArgumentException("The predicate and transformer
arrays must be the same size");
}
if (predicates.length == 0) {
return (Transformer) (defaultTransformer == null ?
ConstantTransformer.nullTransformer() :
defaultTransformer);
}
return new SwitchTransformer<>(predicates, transformers,
defaultTransformer);
}
{code}
I'll submit a PR to add the missing @throws documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-collections] Prodigysov opened a new pull request #123: [COLLECTIONS-739] Fix inconsistent @throws comments in DefaultedMap
Prodigysov opened a new pull request #123: [COLLECTIONS-739] Fix inconsistent @throws comments in DefaultedMap URL: https://github.com/apache/commons-collections/pull/123 See [COLLECTIONS-739](https://issues.apache.org/jira/browse/COLLECTIONS-739). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (COLLECTIONS-739) Wrong @throws comments in DefaultedMap
[
https://issues.apache.org/jira/browse/COLLECTIONS-739?focusedWorklogId=353846&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-353846
]
ASF GitHub Bot logged work on COLLECTIONS-739:
--
Author: ASF GitHub Bot
Created on: 04/Dec/19 22:12
Start Date: 04/Dec/19 22:12
Worklog Time Spent: 10m
Work Description: Prodigysov commented on pull request #123:
[COLLECTIONS-739] Fix inconsistent @throws comments in DefaultedMap
URL: https://github.com/apache/commons-collections/pull/123
See [COLLECTIONS-739](https://issues.apache.org/jira/browse/COLLECTIONS-739).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 353846)
Remaining Estimate: 0h
Time Spent: 10m
> Wrong @throws comments in DefaultedMap
> --
>
> Key: COLLECTIONS-739
> URL: https://issues.apache.org/jira/browse/COLLECTIONS-739
> Project: Commons Collections
> Issue Type: Improvement
> Components: Map
>Reporter: Pengyu Nie
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I found some @throws comments in DefaultedMap are inconsistent with the code
> or missing. For examples:
> 1. The comment for defaultedMap(Map, Factory) has wrong
> exception type:
> Line 108:
> {code:java}
> /** ... @throws NullPointerException if map or factory is null ... */
> {code}
> Line 112-114:
> {code:java}
> if (factory == null) {
> throw new IllegalArgumentException("Factory must not be null");
> }
> {code}
> 2. For defaultMap(Map, Transformer, this null
> checking does not have corresponding @throws comment:
> Line 135-137:
> {code:java}
> if (transformer == null) {
> throw new IllegalArgumentException("Transformer must not be null");
> }
> {code}
> I suggest adding:
> {code:java}
> @throws IllegalArgumentException if transformer is null.
> {code}
> I'll submit a PR for these suggested changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (COLLECTIONS-739) Wrong @throws comments in DefaultedMap
Pengyu Nie created COLLECTIONS-739:
--
Summary: Wrong @throws comments in DefaultedMap
Key: COLLECTIONS-739
URL: https://issues.apache.org/jira/browse/COLLECTIONS-739
Project: Commons Collections
Issue Type: Improvement
Components: Map
Reporter: Pengyu Nie
I found some @throws comments in DefaultedMap are inconsistent with the code or
missing. For examples:
1. The comment for defaultedMap(Map, Factory) has wrong
exception type:
Line 108:
{code:java}
/** ... @throws NullPointerException if map or factory is null ... */
{code}
Line 112-114:
{code:java}
if (factory == null) {
throw new IllegalArgumentException("Factory must not be null");
}
{code}
2. For defaultMap(Map, Transformer, this null
checking does not have corresponding @throws comment:
Line 135-137:
{code:java}
if (transformer == null) {
throw new IllegalArgumentException("Transformer must not be null");
}
{code}
I suggest adding:
{code:java}
@throws IllegalArgumentException if transformer is null.
{code}
I'll submit a PR for these suggested changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (NUMBERS-136) "log10" inconsistent with "log"
[
https://issues.apache.org/jira/browse/NUMBERS-136?focusedWorklogId=353784&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-353784
]
ASF GitHub Bot logged work on NUMBERS-136:
--
Author: ASF GitHub Bot
Created on: 04/Dec/19 21:18
Start Date: 04/Dec/19 21:18
Worklog Time Spent: 10m
Work Description: ericbarnhill commented on pull request #71:
NUMBERS-136: log10() uses log() functionality
URL: https://github.com/apache/commons-numbers/pull/71
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 353784)
Time Spent: 40m (was: 0.5h)
> "log10" inconsistent with "log"
> ---
>
> Key: NUMBERS-136
> URL: https://issues.apache.org/jira/browse/NUMBERS-136
> Project: Commons Numbers
> Issue Type: Bug
> Components: complex
>Reporter: Gilles Sadowski
>Assignee: Eric Barnhill
>Priority: Major
> Fix For: 1.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> For special cases (containing NaN or infinities), method {{log10}} does not
> return the same values as {{log()}} (cf. case commented out below).
> {code}
> @Test
> public void testLog10Consistency() {
> // Infinities should map to the same values as computed with "log()".
> // Assertions.assertEquals(infInf.log(), infInf.log10());
> Assertions.assertEquals(infNegInf.log(), infNegInf.log10());
> Assertions.assertEquals(negInfInf.log(), negInfInf.log10());
> // Assertions.assertEquals(infNaN.log(), infNaN.log10());
> Assertions.assertEquals(negInfNaN.log(), negInfNaN.log10());
> // Assertions.assertEquals(nanInf.log(), nanInf.log10());
> Assertions.assertEquals(nanNegInf.log(), nanNegInf.log10());
> }
> {code}
> To avoid duplication, I suggest that {{log10}} calls {{log}} as part of its
> implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (NUMBERS-136) "log10" inconsistent with "log"
[
https://issues.apache.org/jira/browse/NUMBERS-136?focusedWorklogId=353783&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-353783
]
ASF GitHub Bot logged work on NUMBERS-136:
--
Author: ASF GitHub Bot
Created on: 04/Dec/19 21:18
Start Date: 04/Dec/19 21:18
Worklog Time Spent: 10m
Work Description: ericbarnhill commented on issue #71: NUMBERS-136:
log10() uses log() functionality
URL: https://github.com/apache/commons-numbers/pull/71#issuecomment-561842987
You are right, I thought I pulled in the latest master before I began work,
but somehow did not
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 353783)
Time Spent: 0.5h (was: 20m)
> "log10" inconsistent with "log"
> ---
>
> Key: NUMBERS-136
> URL: https://issues.apache.org/jira/browse/NUMBERS-136
> Project: Commons Numbers
> Issue Type: Bug
> Components: complex
>Reporter: Gilles Sadowski
>Assignee: Eric Barnhill
>Priority: Major
> Fix For: 1.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> For special cases (containing NaN or infinities), method {{log10}} does not
> return the same values as {{log()}} (cf. case commented out below).
> {code}
> @Test
> public void testLog10Consistency() {
> // Infinities should map to the same values as computed with "log()".
> // Assertions.assertEquals(infInf.log(), infInf.log10());
> Assertions.assertEquals(infNegInf.log(), infNegInf.log10());
> Assertions.assertEquals(negInfInf.log(), negInfInf.log10());
> // Assertions.assertEquals(infNaN.log(), infNaN.log10());
> Assertions.assertEquals(negInfNaN.log(), negInfNaN.log10());
> // Assertions.assertEquals(nanInf.log(), nanInf.log10());
> Assertions.assertEquals(nanNegInf.log(), nanNegInf.log10());
> }
> {code}
> To avoid duplication, I suggest that {{log10}} calls {{log}} as part of its
> implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-numbers] ericbarnhill commented on issue #71: NUMBERS-136: log10() uses log() functionality
ericbarnhill commented on issue #71: NUMBERS-136: log10() uses log() functionality URL: https://github.com/apache/commons-numbers/pull/71#issuecomment-561842987 You are right, I thought I pulled in the latest master before I began work, but somehow did not This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [commons-numbers] ericbarnhill closed pull request #71: NUMBERS-136: log10() uses log() functionality
ericbarnhill closed pull request #71: NUMBERS-136: log10() uses log() functionality URL: https://github.com/apache/commons-numbers/pull/71 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (NUMBERS-136) "log10" inconsistent with "log"
[
https://issues.apache.org/jira/browse/NUMBERS-136?focusedWorklogId=353771&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-353771
]
ASF GitHub Bot logged work on NUMBERS-136:
--
Author: ASF GitHub Bot
Created on: 04/Dec/19 21:08
Start Date: 04/Dec/19 21:08
Worklog Time Spent: 10m
Work Description: aherbert commented on issue #71: NUMBERS-136: log10()
uses log() functionality
URL: https://github.com/apache/commons-numbers/pull/71#issuecomment-561839176
Hi @ericbarnhill, I think you have based this on an old version of master.
I fixed log10 when fixing the ISO C99 reference test. I changed the
implementation to use Math.log10 in place of Math.log.
Have a look at the current master.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 353771)
Time Spent: 20m (was: 10m)
> "log10" inconsistent with "log"
> ---
>
> Key: NUMBERS-136
> URL: https://issues.apache.org/jira/browse/NUMBERS-136
> Project: Commons Numbers
> Issue Type: Bug
> Components: complex
>Reporter: Gilles Sadowski
>Assignee: Eric Barnhill
>Priority: Major
> Fix For: 1.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For special cases (containing NaN or infinities), method {{log10}} does not
> return the same values as {{log()}} (cf. case commented out below).
> {code}
> @Test
> public void testLog10Consistency() {
> // Infinities should map to the same values as computed with "log()".
> // Assertions.assertEquals(infInf.log(), infInf.log10());
> Assertions.assertEquals(infNegInf.log(), infNegInf.log10());
> Assertions.assertEquals(negInfInf.log(), negInfInf.log10());
> // Assertions.assertEquals(infNaN.log(), infNaN.log10());
> Assertions.assertEquals(negInfNaN.log(), negInfNaN.log10());
> // Assertions.assertEquals(nanInf.log(), nanInf.log10());
> Assertions.assertEquals(nanNegInf.log(), nanNegInf.log10());
> }
> {code}
> To avoid duplication, I suggest that {{log10}} calls {{log}} as part of its
> implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [commons-numbers] aherbert commented on issue #71: NUMBERS-136: log10() uses log() functionality
aherbert commented on issue #71: NUMBERS-136: log10() uses log() functionality URL: https://github.com/apache/commons-numbers/pull/71#issuecomment-561839176 Hi @ericbarnhill, I think you have based this on an old version of master. I fixed log10 when fixing the ISO C99 reference test. I changed the implementation to use Math.log10 in place of Math.log. Have a look at the current master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (BEANUTILS-524) Make BeanUtils copyProperties able to copy Immutable, Mutable and Mixed Java Bean
[ https://issues.apache.org/jira/browse/BEANUTILS-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988156#comment-16988156 ] Fabio Borriello commented on BEANUTILS-524: --- I’ll work on that > Make BeanUtils copyProperties able to copy Immutable, Mutable and Mixed Java > Bean > - > > Key: BEANUTILS-524 > URL: https://issues.apache.org/jira/browse/BEANUTILS-524 > Project: Commons BeanUtils > Issue Type: New Feature > Components: Locale BeanUtils / Converters >Reporter: Fabio Borriello >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > h4. Overview > the copyProperties method in the BeanUtils class, is able to copy only public > properties or the ones with a public setter method. > it would be really useful if this functionality can be extended in order to > copy any kind of Java Bean: Mutable, Immutable, Mixed. > h4. Solution > Evaluate the possibility to use, behind the scenes, an open source library > that is able to copy Mutable, Immutable and Mixed Java Bean. The library In > object is [BULL|https://github.com/HotelsDotCom/bull]. > This would also make able to copy nested objects. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NUMBERS-136) "log10" inconsistent with "log"
[
https://issues.apache.org/jira/browse/NUMBERS-136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988143#comment-16988143
]
Eric Barnhill commented on NUMBERS-136:
---
Work submitted as PR 71
> "log10" inconsistent with "log"
> ---
>
> Key: NUMBERS-136
> URL: https://issues.apache.org/jira/browse/NUMBERS-136
> Project: Commons Numbers
> Issue Type: Bug
> Components: complex
>Reporter: Gilles Sadowski
>Assignee: Eric Barnhill
>Priority: Major
> Fix For: 1.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For special cases (containing NaN or infinities), method {{log10}} does not
> return the same values as {{log()}} (cf. case commented out below).
> {code}
> @Test
> public void testLog10Consistency() {
> // Infinities should map to the same values as computed with "log()".
> // Assertions.assertEquals(infInf.log(), infInf.log10());
> Assertions.assertEquals(infNegInf.log(), infNegInf.log10());
> Assertions.assertEquals(negInfInf.log(), negInfInf.log10());
> // Assertions.assertEquals(infNaN.log(), infNaN.log10());
> Assertions.assertEquals(negInfNaN.log(), negInfNaN.log10());
> // Assertions.assertEquals(nanInf.log(), nanInf.log10());
> Assertions.assertEquals(nanNegInf.log(), nanNegInf.log10());
> }
> {code}
> To avoid duplication, I suggest that {{log10}} calls {{log}} as part of its
> implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CRYPTO-143) Release a new version to maven central
[ https://issues.apache.org/jira/browse/CRYPTO-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988135#comment-16988135 ] James Baker commented on CRYPTO-143: Gary, thanks. It's sad that no release has happened because there's a _lot_ of code on trunk that's unreleased (which presumably has never been used) and this project does not appear to publish snapshots. In our case we fall back to slow JCE if commons crypto fails to initialize for some reason and it was sad to realise that we'd deoptimized ourselves by upgrading Ubuntu! Appreciate the work of volunteers on this codebase and others. > Release a new version to maven central > -- > > Key: CRYPTO-143 > URL: https://issues.apache.org/jira/browse/CRYPTO-143 > Project: Commons Crypto > Issue Type: Task >Reporter: Chris >Priority: Major > > I've been maintaining my own fork of commons-crypto for over 2 years, shortly > after forking someone merged the change I wanted into the master branch. > > It would be nice to no longer have to maintain or build my fork, or even > build the master branch myself. There have been a number of fixes and > enhancements in the last 2 years... Could we get a 1.0.1 release to maven > central? > > Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (CRYPTO-143) Release a new version to maven central
[ https://issues.apache.org/jira/browse/CRYPTO-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988135#comment-16988135 ] James Baker edited comment on CRYPTO-143 at 12/4/19 7:37 PM: - Gary, thanks. It's sad that no release has happened because there's a _lot_ of code on trunk that's unreleased (which presumably has never been used) and this project does not appear to publish snapshots. In our case we fall back to slow JCE if commons crypto fails to initialize for some reason and it was sad to realise that we'd deoptimized ourselves by upgrading Ubuntu! Appreciate the work of volunteers on this codebase and others - it's enabled a significant performance improvement for us. was (Author: jebaker): Gary, thanks. It's sad that no release has happened because there's a _lot_ of code on trunk that's unreleased (which presumably has never been used) and this project does not appear to publish snapshots. In our case we fall back to slow JCE if commons crypto fails to initialize for some reason and it was sad to realise that we'd deoptimized ourselves by upgrading Ubuntu! Appreciate the work of volunteers on this codebase and others. > Release a new version to maven central > -- > > Key: CRYPTO-143 > URL: https://issues.apache.org/jira/browse/CRYPTO-143 > Project: Commons Crypto > Issue Type: Task >Reporter: Chris >Priority: Major > > I've been maintaining my own fork of commons-crypto for over 2 years, shortly > after forking someone merged the change I wanted into the master branch. > > It would be nice to no longer have to maintain or build my fork, or even > build the master branch myself. There have been a number of fixes and > enhancements in the last 2 years... Could we get a 1.0.1 release to maven > central? > > Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (CRYPTO-143) Release a new version to maven central
[ https://issues.apache.org/jira/browse/CRYPTO-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988125#comment-16988125 ] Gary D. Gregory commented on CRYPTO-143: [~jebaker] We are all volunteers here, we do the best we can and some components get less attention... hopefully someone will step in to RM... > Release a new version to maven central > -- > > Key: CRYPTO-143 > URL: https://issues.apache.org/jira/browse/CRYPTO-143 > Project: Commons Crypto > Issue Type: Task >Reporter: Chris >Priority: Major > > I've been maintaining my own fork of commons-crypto for over 2 years, shortly > after forking someone merged the change I wanted into the master branch. > > It would be nice to no longer have to maintain or build my fork, or even > build the master branch myself. There have been a number of fixes and > enhancements in the last 2 years... Could we get a 1.0.1 release to maven > central? > > Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [commons-beanutils] garydgregory commented on a change in pull request #20: [BEANUTILS-524] Copy properties improvement
garydgregory commented on a change in pull request #20: [BEANUTILS-524] Copy properties improvement URL: https://github.com/apache/commons-beanutils/pull/20#discussion_r353938282 ## File path: pom.xml ## @@ -341,6 +346,11 @@ 4.12 test + + com.hotels.beans + bull-bean-transformer Review comment: Hey @jvz you're little behind ;-) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [commons-numbers] ericbarnhill opened a new pull request #71: NUMBERS-136: log10() uses log() functionality
ericbarnhill opened a new pull request #71: NUMBERS-136: log10() uses log() functionality URL: https://github.com/apache/commons-numbers/pull/71 log10() greatly simplified and uses log() functionality This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work logged] (NUMBERS-136) "log10" inconsistent with "log"
[
https://issues.apache.org/jira/browse/NUMBERS-136?focusedWorklogId=353666&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-353666
]
ASF GitHub Bot logged work on NUMBERS-136:
--
Author: ASF GitHub Bot
Created on: 04/Dec/19 19:05
Start Date: 04/Dec/19 19:05
Worklog Time Spent: 10m
Work Description: ericbarnhill commented on pull request #71:
NUMBERS-136: log10() uses log() functionality
URL: https://github.com/apache/commons-numbers/pull/71
log10() greatly simplified and uses log() functionality
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 353666)
Remaining Estimate: 0h
Time Spent: 10m
> "log10" inconsistent with "log"
> ---
>
> Key: NUMBERS-136
> URL: https://issues.apache.org/jira/browse/NUMBERS-136
> Project: Commons Numbers
> Issue Type: Bug
> Components: complex
>Reporter: Gilles Sadowski
>Assignee: Eric Barnhill
>Priority: Major
> Fix For: 1.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For special cases (containing NaN or infinities), method {{log10}} does not
> return the same values as {{log()}} (cf. case commented out below).
> {code}
> @Test
> public void testLog10Consistency() {
> // Infinities should map to the same values as computed with "log()".
> // Assertions.assertEquals(infInf.log(), infInf.log10());
> Assertions.assertEquals(infNegInf.log(), infNegInf.log10());
> Assertions.assertEquals(negInfInf.log(), negInfInf.log10());
> // Assertions.assertEquals(infNaN.log(), infNaN.log10());
> Assertions.assertEquals(negInfNaN.log(), negInfNaN.log10());
> // Assertions.assertEquals(nanInf.log(), nanInf.log10());
> Assertions.assertEquals(nanNegInf.log(), nanNegInf.log10());
> }
> {code}
> To avoid duplication, I suggest that {{log10}} calls {{log}} as part of its
> implementation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CRYPTO-143) Release a new version to maven central
[ https://issues.apache.org/jira/browse/CRYPTO-143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16988080#comment-16988080 ] James Baker commented on CRYPTO-143: Would also appreciate a release here. No release since 2016 - is this project dead? > Release a new version to maven central > -- > > Key: CRYPTO-143 > URL: https://issues.apache.org/jira/browse/CRYPTO-143 > Project: Commons Crypto > Issue Type: Task >Reporter: Chris >Priority: Major > > I've been maintaining my own fork of commons-crypto for over 2 years, shortly > after forking someone merged the change I wanted into the master branch. > > It would be nice to no longer have to maintain or build my fork, or even > build the master branch myself. There have been a number of fixes and > enhancements in the last 2 years... Could we get a 1.0.1 release to maven > central? > > Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [commons-beanutils] melloware commented on a change in pull request #20: [BEANUTILS-524] Copy properties improvement
melloware commented on a change in pull request #20: [BEANUTILS-524] Copy properties improvement URL: https://github.com/apache/commons-beanutils/pull/20#discussion_r353892818 ## File path: pom.xml ## @@ -341,6 +346,11 @@ 4.12 test + + com.hotels.beans + bull-bean-transformer Review comment: @jvz you can read our long thread about this in the JIRA ticket. I said the same thing and proposed him some alternatives. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [commons-beanutils] jvz commented on a change in pull request #20: [BEANUTILS-524] Copy properties improvement
jvz commented on a change in pull request #20: [BEANUTILS-524] Copy properties improvement URL: https://github.com/apache/commons-beanutils/pull/20#discussion_r353890349 ## File path: pom.xml ## @@ -341,6 +346,11 @@ 4.12 test + + com.hotels.beans + bull-bean-transformer Review comment: Is this for testing? I don't think Commons components typically have compile dependencies besides other Commons projects usually. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (POOL-382) Release 2.7.1
[ https://issues.apache.org/jira/browse/POOL-382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16987965#comment-16987965 ] Gary D. Gregory commented on POOL-382: -- Hi [~sergtitov] This is a not the right venue for requesting a release, please use the user or dev mailing list. Gary > Release 2.7.1 > - > > Key: POOL-382 > URL: https://issues.apache.org/jira/browse/POOL-382 > Project: Commons Pool > Issue Type: Task >Affects Versions: 2.7.0 >Reporter: Sergei Titov >Priority: Major > Labels: release > > Please release 2.7.1. > The latest version 2.7.0 was released in July and since then there were bug > fixes committed to master that were never released. We are especially > interested in POOL-376 which happens in Prod for us periodically. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [commons-collections] garydgregory commented on a change in pull request #122: Improve MapUtils with the null checks, add Javadoc for the parameter indent and add JUnit for it.
garydgregory commented on a change in pull request #122: Improve MapUtils with
the null checks, add Javadoc for the parameter indent and add JUnit for it.
URL:
https://github.com/apache/commons-collections/pull/122#discussion_r353829545
##
File path: src/main/java/org/apache/commons/collections4/MapUtils.java
##
@@ -1186,11 +1188,12 @@ public static void debugPrint(final PrintStream out,
final Object label, final M
*
* @param the key type
* @param the value type
- * @param map the map to invert, may not be null
+ * @param map the map to invert, must not be null
* @return a new HashMap containing the inverted data
* @throws NullPointerException if the map is null
*/
public static Map invertMap(final Map map) {
+Objects.requireNonNull(map, "The map must not be null");
Review comment:
Please simply use the argument name like "map" or whatever the faulty
argument name is.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (CODEC-272) Add RandomAccessFile digest methods #31
[ https://issues.apache.org/jira/browse/CODEC-272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary D. Gregory resolved CODEC-272. --- Fix Version/s: 1.14 Resolution: Fixed In git master, please verify and close. - This is a slightly different version from https://github.com/apache/commons-codec/pull/31/ - Refactor updateDigest(MessageDigest,RandomAccessFile) into an new private updateDigest(MessageDigest,FileChannel) as possible public candidate. - Do NOT seek to 0 on a RandomAccessFile before calling updateDigest(): We do not do this for ByteBuffer input, so do not do it here and be consistent to assume that when the caller says 'digest this' then do it from where the input stands (like a stream). - Add methods in the file to keep methods in alphabetical order. > Add RandomAccessFile digest methods #31 > --- > > Key: CODEC-272 > URL: https://issues.apache.org/jira/browse/CODEC-272 > Project: Commons Codec > Issue Type: New Feature >Reporter: Gary D. Gregory >Assignee: Gary D. Gregory >Priority: Major > Fix For: 1.14 > > > Add RandomAccessFile digest methods #31. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [commons-codec] asfgit closed pull request #31: Add RandomAccessFile digest methods
asfgit closed pull request #31: Add RandomAccessFile digest methods URL: https://github.com/apache/commons-codec/pull/31 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (CODEC-272) Add RandomAccessFile digest methods #31
Gary D. Gregory created CODEC-272: - Summary: Add RandomAccessFile digest methods #31 Key: CODEC-272 URL: https://issues.apache.org/jira/browse/CODEC-272 Project: Commons Codec Issue Type: New Feature Reporter: Gary D. Gregory Add RandomAccessFile digest methods #31. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (CODEC-272) Add RandomAccessFile digest methods #31
[ https://issues.apache.org/jira/browse/CODEC-272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary D. Gregory updated CODEC-272: -- Assignee: Gary D. Gregory > Add RandomAccessFile digest methods #31 > --- > > Key: CODEC-272 > URL: https://issues.apache.org/jira/browse/CODEC-272 > Project: Commons Codec > Issue Type: New Feature >Reporter: Gary D. Gregory >Assignee: Gary D. Gregory >Priority: Major > > Add RandomAccessFile digest methods #31. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [commons-codec] garydgregory commented on issue #31: Add RandomAccessFile digest methods
garydgregory commented on issue #31: Add RandomAccessFile digest methods URL: https://github.com/apache/commons-codec/pull/31#issuecomment-561704159 xref https://issues.apache.org/jira/browse/CODEC-272 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
