[jira] [Comment Edited] (COMPRESS-502) Allow to disable closing files in the finalizer of ZipFile

[Dominik Stadler (Jira)]

[ 
https://issues.apache.org/jira/browse/COMPRESS-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023640#comment-17023640
 ] 

Dominik Stadler edited comment on COMPRESS-502 at 1/26/20 7:49 AM:
---

We would like to use another tool to better report the actual location of where 
the file was opened as the single output to stderr can be too less to find the 
location of allocation in a fairly sized application.

So I mostly would like to not have "close()" be run as part of the finalizer.

The output to stderr is another shortcoming which might be unwanted for others.

A simple solution as far as I see would be to make it pluggable via a simple 
static setter and use the current System.err.println as default.

 

E.g. something like the following (did not compile this, so might not fully 
work this way):

 
{noformat}
private static Function finalizeFun = (file) -> {
if (!closed) {
    System.err.println("Cleaning up unclosed ZipFile for archive " + 
archiveName);
        file.close();
}
 };

public static void setFinalizer(Function fun) {
    finalizeFun = fun;
}

protected void finalize() throws Throwable {
    try {
        finalizeRunnable->apply(this);
    } finally {
   super.finalize();
    }
}
{noformat}


was (Author: dominik.stad...@gmx.at):
We would like to use another tool to better report the actual location of where 
the file was opened as the single output to stderr can be too less to find the 
location of allocation in a fairly sized application.

So I mostly would like to not have "close()" be run as part of the finalizer.

The output to stderr is another shortcoming which might be unwanted for others.

A simple solution as far as I see would be to make it pluggable via a simple 
static setter and use the current System.err.println as default.

 

E.g. something like the following (did not compile this, so might not fully 
work this way):

 
{noformat}
private static Function finalizeFun = (file) -> {
    if (!closed) {
    System.err.println("Cleaning up unclosed ZipFile for archive " + 
archiveName);
            file.close();
    }
 };

public static void setFinalizer(Function fun) {
    finalizeFun = fun;
}

protected void finalize() throws Throwable {
    try {
        finalizeRunnable->apply(this);
    } finally {
   super.finalize();
    }
}
{noformat}

> Allow to disable closing files in the finalizer of ZipFile
> --
>
> Key: COMPRESS-502
> URL: https://issues.apache.org/jira/browse/COMPRESS-502
> Project: Commons Compress
>  Issue Type: Improvement
>  Components: Compressors
>Affects Versions: 1.19
>Reporter: Dominik Stadler
>Priority: Major
>
> Apache POI uses commons-compress for handling ZipFiles. We found that it 
> sometimes does some auto-close magic in the finalizer of the ZipFile class 
> with printing out to stderr, see 
> https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=blob;f=src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java;h=23194560dace91d8052626f3bdc8f765f9c46d7e;hb=HEAD#l652.
>  
> This has some shortcomings:
>  * It prints to stderr which many large-scale applications try to avoid by 
> using some logging framework, thus this output might "vanish" unseen in some 
> installations or might cause unexpected side-effects
>  * It prevents us from using tools for checking file leaks, e.g. we use 
> [https://github.com/kohsuke/file-leak-detector/] heavily for analyzing 
> test-runs for leaked file-handles, but commons-compress prevents this because 
> it "hides" the unclosed file from this functionality
>  * The behavior of automatic closing and reporting the problem is 
> non-reproducible because it depends on finalizer/garbage-collection and thus 
> any re-runs or unit-tests usually do not show the same behavior
>  
> There are some fairly simple options to improve this:
>  * Allow to disable this functionality via configuration/system-property/...
>  * Make this "pluggable" so a logging-framework can be plugged-in or closing 
> can be prevented for certain runs
>  
> I can provide a simple patch if you state which approach you think would make 
> most sense here.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (COMPRESS-502) Allow to disable closing files in the finalizer of ZipFile

[Dominik Stadler (Jira)]

[ 
https://issues.apache.org/jira/browse/COMPRESS-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023640#comment-17023640
 ] 

Dominik Stadler edited comment on COMPRESS-502 at 1/26/20 7:49 AM:
---

We would like to use another tool to better report the actual location of where 
the file was opened as the single output to stderr can be too less to find the 
location of allocation in a fairly sized application.

So I mostly would like to not have "close()" be run as part of the finalizer.

The output to stderr is another shortcoming which might be unwanted for others.

A simple solution as far as I see would be to make it pluggable via a simple 
static setter and use the current System.err.println as default.

 

E.g. something like the following (did not compile this, so might not fully 
work this way):

 
{noformat}
private static Function finalizeFun = (file) -> {
if (!closed) {
    System.err.println("Cleaning up unclosed ZipFile for archive " + 
archiveName);
        file.close();
}
 };

public static void setFinalizer(Function fun) {
    finalizeFun = fun;
}

protected void finalize() throws Throwable {
    try {
        finalizeFun->apply(this);
    } finally {
   super.finalize();
    }
}
{noformat}


was (Author: dominik.stad...@gmx.at):
We would like to use another tool to better report the actual location of where 
the file was opened as the single output to stderr can be too less to find the 
location of allocation in a fairly sized application.

So I mostly would like to not have "close()" be run as part of the finalizer.

The output to stderr is another shortcoming which might be unwanted for others.

A simple solution as far as I see would be to make it pluggable via a simple 
static setter and use the current System.err.println as default.

 

E.g. something like the following (did not compile this, so might not fully 
work this way):

 
{noformat}
private static Function finalizeFun = (file) -> {
if (!closed) {
    System.err.println("Cleaning up unclosed ZipFile for archive " + 
archiveName);
        file.close();
}
 };

public static void setFinalizer(Function fun) {
    finalizeFun = fun;
}

protected void finalize() throws Throwable {
    try {
        finalizeRunnable->apply(this);
    } finally {
   super.finalize();
    }
}
{noformat}

> Allow to disable closing files in the finalizer of ZipFile
> --
>
> Key: COMPRESS-502
> URL: https://issues.apache.org/jira/browse/COMPRESS-502
> Project: Commons Compress
>  Issue Type: Improvement
>  Components: Compressors
>Affects Versions: 1.19
>Reporter: Dominik Stadler
>Priority: Major
>
> Apache POI uses commons-compress for handling ZipFiles. We found that it 
> sometimes does some auto-close magic in the finalizer of the ZipFile class 
> with printing out to stderr, see 
> https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=blob;f=src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java;h=23194560dace91d8052626f3bdc8f765f9c46d7e;hb=HEAD#l652.
>  
> This has some shortcomings:
>  * It prints to stderr which many large-scale applications try to avoid by 
> using some logging framework, thus this output might "vanish" unseen in some 
> installations or might cause unexpected side-effects
>  * It prevents us from using tools for checking file leaks, e.g. we use 
> [https://github.com/kohsuke/file-leak-detector/] heavily for analyzing 
> test-runs for leaked file-handles, but commons-compress prevents this because 
> it "hides" the unclosed file from this functionality
>  * The behavior of automatic closing and reporting the problem is 
> non-reproducible because it depends on finalizer/garbage-collection and thus 
> any re-runs or unit-tests usually do not show the same behavior
>  
> There are some fairly simple options to improve this:
>  * Allow to disable this functionality via configuration/system-property/...
>  * Make this "pluggable" so a logging-framework can be plugged-in or closing 
> can be prevented for certain runs
>  
> I can provide a simple patch if you state which approach you think would make 
> most sense here.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (COMPRESS-502) Allow to disable closing files in the finalizer of ZipFile

[Dominik Stadler (Jira)]

[ 
https://issues.apache.org/jira/browse/COMPRESS-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023640#comment-17023640
 ] 

Dominik Stadler edited comment on COMPRESS-502 at 1/26/20 7:48 AM:
---

We would like to use another tool to better report the actual location of where 
the file was opened as the single output to stderr can be too less to find the 
location of allocation in a fairly sized application.

So I mostly would like to not have "close()" be run as part of the finalizer.

The output to stderr is another shortcoming which might be unwanted for others.

A simple solution as far as I see would be to make it pluggable via a simple 
static setter and use the current System.err.println as default.

 

E.g. something like the following (did not compile this, so might not fully 
work this way):

 
{noformat}
private static Function finalizeFun = (file) -> {
    if (!closed) {
    System.err.println("Cleaning up unclosed ZipFile for archive " + 
archiveName);
            file.close();
    }
 };

public static void setFinalizer(Function fun) {
    finalizeFun = fun;
}

protected void finalize() throws Throwable {
    try {
        finalizeRunnable->apply(this);
    } finally {
   super.finalize();
    }
}
{noformat}


was (Author: dominik.stad...@gmx.at):
We would like to use another tool to better report the actual location of where 
the file was opened as the single output to stderr can be too less to find the 
location of allocation in a fairly sized application.

So I mostly would like to not have "close()" be run as part of the finalizer.

The output to stderr is another shortcoming which might be unwanted for others.

A simple solution as far as I see would be to make it pluggable via a simple 
static setter and use the current System.err.println as default.

 

E.g. something like the following (did not compile this, so might not fully 
work this way):

 

private static Function finalizeFun = (file) -> {

    if (!closed) {
    System.err.println("Cleaning up unclosed ZipFile for archive " + 
archiveName);
            file.close();
    }
 };

public static void setFinalizer(Function fun) {

    finalizeFun = fun;

}

protected void finalize() throws Throwable {

    try {

        finalizeRunnable->apply(this);

    } finally {
   super.finalize();
    }

}

 

> Allow to disable closing files in the finalizer of ZipFile
> --
>
> Key: COMPRESS-502
> URL: https://issues.apache.org/jira/browse/COMPRESS-502
> Project: Commons Compress
>  Issue Type: Improvement
>  Components: Compressors
>Affects Versions: 1.19
>Reporter: Dominik Stadler
>Priority: Major
>
> Apache POI uses commons-compress for handling ZipFiles. We found that it 
> sometimes does some auto-close magic in the finalizer of the ZipFile class 
> with printing out to stderr, see 
> https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=blob;f=src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java;h=23194560dace91d8052626f3bdc8f765f9c46d7e;hb=HEAD#l652.
>  
> This has some shortcomings:
>  * It prints to stderr which many large-scale applications try to avoid by 
> using some logging framework, thus this output might "vanish" unseen in some 
> installations or might cause unexpected side-effects
>  * It prevents us from using tools for checking file leaks, e.g. we use 
> [https://github.com/kohsuke/file-leak-detector/] heavily for analyzing 
> test-runs for leaked file-handles, but commons-compress prevents this because 
> it "hides" the unclosed file from this functionality
>  * The behavior of automatic closing and reporting the problem is 
> non-reproducible because it depends on finalizer/garbage-collection and thus 
> any re-runs or unit-tests usually do not show the same behavior
>  
> There are some fairly simple options to improve this:
>  * Allow to disable this functionality via configuration/system-property/...
>  * Make this "pluggable" so a logging-framework can be plugged-in or closing 
> can be prevented for certain runs
>  
> I can provide a simple patch if you state which approach you think would make 
> most sense here.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [commons-dbutils] thecarlhall closed pull request #7: get generated keys from queryRunner.insertBatch

[GitBox]

thecarlhall closed pull request #7: get generated keys from 
queryRunner.insertBatch
URL: https://github.com/apache/commons-dbutils/pull/7
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [commons-dbutils] thecarlhall commented on issue #7: get generated keys from queryRunner.insertBatch

[GitBox]

thecarlhall commented on issue #7: get generated keys from 
queryRunner.insertBatch
URL: https://github.com/apache/commons-dbutils/pull/7#issuecomment-578471964
 
 
   I'm glad to hear this works for you, @visruth!
   Your PR highlights the need for more docs. I'll work to update the examples.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [commons-dbutils] visruth edited a comment on issue #7: get generated keys from queryRunner.insertBatch

[GitBox]

visruth edited a comment on issue #7: get generated keys from 
queryRunner.insertBatch
URL: https://github.com/apache/commons-dbutils/pull/7#issuecomment-578377584
 
 
   Hi @thecarlhall, `queryRunner.insertBatch` doesn't support 
`ColumnListHandler`. How can I get generated keys (primary key) of batch insert 
with `ColumnListHandler`?
   If you look in to the sample code I have given you can see that `ids` 
contain the generated keys of inserted rows in the same order of `params` rows 
inserted.
   
   For multiple rows insertion in to same table I'm using batch insert in my 
project which reduces a lot of time compared to individual insert operation.
   
   **NB: update**
   `queryRunner.insertBatch` does support `ColumnListHandler` that also 
achieves the same goal.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [commons-dbutils] visruth commented on issue #7: get generated keys from queryRunner.insertBatch

[GitBox]

visruth commented on issue #7: get generated keys from queryRunner.insertBatch
URL: https://github.com/apache/commons-dbutils/pull/7#issuecomment-578467084
 
 
   Hi @thecarlhall, It's my mistake, `queryRunner.insertBatch` does support 
`ColumnListHandler` that also achieves the same goal. Yes, it meets my need.
   When I first got this requirement I made a lot of research to find if there 
is any existing class available to achieve the same but didn't find any so I 
implemented `ScalarListHandler` its name can be easily guessed by the developer 
because there is already a class named `ScalarHandler` which returns only one 
value.
   Thanks a lot @thecarlhall for notifying this.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (CSV-257) Updating from 1.6 to 1.7 breaks

[xia0c (Jira)]

xia0c created CSV-257:
-

 Summary: Updating from 1.6 to 1.7 breaks
 Key: CSV-257
 URL: https://issues.apache.org/jira/browse/CSV-257
 Project: Commons CSV
  Issue Type: Bug
  Components: Parser
Affects Versions: 1.7
Reporter: xia0c


When I try to upgrade commons csv from 1.6 to the latest version 1.7. The 
following code breaks.

{code:java}
public class Demo {

@Test
public void TestCSV(){
try {
InputStream testInput = new 
ByteArrayInputStream(",".getBytes(StandardCharsets.UTF_8));
Reader reader = new InputStreamReader(testInput);
char character = ",".charAt(0);
CSVFormat format = 
CSVFormat.RFC4180.withDelimiter(character).withFirstRecordAsHeader().withIgnoreSurroundingSpaces();
CSVParser parser = new CSVParser(reader, format);

} catch (Exception e) {
System.out.println("!!!");
assertEquals("java.lang.IllegalArgumentException: The 
header contains a duplicate name: \"\" in [, ]", e.toString());
}
}

}
{code}

The test should pass, but it failed with error:

{code:java}
org.junit.ComparisonFailure: expected:<...lArgumentException: [The header 
contains a duplicate name: ""] in [, ]> but was:<...lArgumentException: [A 
header name is missing] in [, ]>
at org.junit.Assert.assertEquals(Assert.java:115)
at org.junit.Assert.assertEquals(Assert.java:144)
at Demo.TestCSV(Demo.java:31)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at 
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:89)
at 
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:41)
at 
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:541)
at 
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:763)
at 
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:463)
at 
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:209)

{code}





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [GitHub] [commons-codec] aherbert commented on a change in pull request #35: [CODEC-280] Added strict decoding property to BaseNCodec.

[Alex Herbert]

> On 26 Jan 2020, at 00:08, Gary Gregory  wrote:
> 
> On Sat, Jan 25, 2020 at 6:12 PM GitBox  > wrote:
> 
>> aherbert commented on a change in pull request #35: [CODEC-280] Added
>> strict decoding property to BaseNCodec.
>> URL: https://github.com/apache/commons-codec/pull/35#discussion_r370961690
>> 
>> 
>> 
>> ##
>> File path: src/test/java/org/apache/commons/codec/net/BCodecTest.java
>> ##
>> @@ -157,6 +158,7 @@ public void testDecodeObjects() throws Exception {
>> }
>> 
>> @Test
>> +@Ignore("CODEC-280: Disabled strict decoding by default. The BCodec
>> uses the default so this test does not fail the impossible cases.")
>> 
>> Review comment:
>>   Hi @garydgregory
>> 
>>   I received an email from github with another comment from you but
>> cannot find it through the web interface so I'll repeat for clarity the
>> comment was "I'm all for making Base32 and Base64 carry the same option."
>> 
>>   This PR already has the public setStrictDecoding(boolean) for Base32
>> and Base64. The unresolved is whether the property should be added to
>> BCodec as well. At the moment this test is ignored because BCodec cannot be
>> made strict. So either:
>> 
>>   1. Drop the test and BCodec is always lenient
>>   2. Make BCodec always strict (this test will pass)
>>   3. Make BCodec have configurable strict/lenient decoding
>> 
>>   My option would be number 3.
>> 
> 
> I'm fine with that.
> Will you do that within the same PR?

Sure. Hopefully I will have time in the next few days.

> 
> Gary
> 
> 
>> 
>> 
>> This is an automated message from the Apache Git Service.
>> To respond to the message, please log on to GitHub and use the
>> URL above to go to the specific comment.
>> 
>> For queries about this service, please contact Infrastructure at:
>> us...@infra.apache.org 
>> 
>> 
>> With regards,
>> Apache Git Services

[GitHub] [commons-dbutils] thecarlhall commented on issue #7: get generated keys from queryRunner.insertBatch

[GitBox]

thecarlhall commented on issue #7: get generated keys from 
queryRunner.insertBatch
URL: https://github.com/apache/commons-dbutils/pull/7#issuecomment-578459850
 
 
   Hi @visruth, I created a [project to show how this can be 
done](https://github.com/thecarlhall/dbutils-pr7test). There's a [test class 
that uses 
ColumnListHandler](https://github.com/thecarlhall/dbutils-pr7test/blob/master/src/test/java/org/apache/commons/dbutils/PR7Test.java)
 to get the IDs generated by an H2 in-memory database. Please let me know if 
this meets need, or if I'm missing something.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Re: [GitHub] [commons-codec] aherbert commented on a change in pull request #35: [CODEC-280] Added strict decoding property to BaseNCodec.

[Gary Gregory]

On Sat, Jan 25, 2020 at 6:12 PM GitBox  wrote:

> aherbert commented on a change in pull request #35: [CODEC-280] Added
> strict decoding property to BaseNCodec.
> URL: https://github.com/apache/commons-codec/pull/35#discussion_r370961690
>
>
>
>  ##
>  File path: src/test/java/org/apache/commons/codec/net/BCodecTest.java
>  ##
>  @@ -157,6 +158,7 @@ public void testDecodeObjects() throws Exception {
>  }
>
>  @Test
> +@Ignore("CODEC-280: Disabled strict decoding by default. The BCodec
> uses the default so this test does not fail the impossible cases.")
>
>  Review comment:
>Hi @garydgregory
>
>I received an email from github with another comment from you but
> cannot find it through the web interface so I'll repeat for clarity the
> comment was "I'm all for making Base32 and Base64 carry the same option."
>
>This PR already has the public setStrictDecoding(boolean) for Base32
> and Base64. The unresolved is whether the property should be added to
> BCodec as well. At the moment this test is ignored because BCodec cannot be
> made strict. So either:
>
>1. Drop the test and BCodec is always lenient
>2. Make BCodec always strict (this test will pass)
>3. Make BCodec have configurable strict/lenient decoding
>
>My option would be number 3.
>

I'm fine with that.
Will you do that within the same PR?

Gary


>
> 
> This is an automated message from the Apache Git Service.
> To respond to the message, please log on to GitHub and use the
> URL above to go to the specific comment.
>
> For queries about this service, please contact Infrastructure at:
> us...@infra.apache.org
>
>
> With regards,
> Apache Git Services
>

[GitHub] [commons-codec] aherbert commented on a change in pull request #35: [CODEC-280] Added strict decoding property to BaseNCodec.

[GitBox]

aherbert commented on a change in pull request #35: [CODEC-280] Added strict 
decoding property to BaseNCodec.
URL: https://github.com/apache/commons-codec/pull/35#discussion_r370961690
 
 

 ##
 File path: src/test/java/org/apache/commons/codec/net/BCodecTest.java
 ##
 @@ -157,6 +158,7 @@ public void testDecodeObjects() throws Exception {
 }
 
 @Test
+@Ignore("CODEC-280: Disabled strict decoding by default. The BCodec uses 
the default so this test does not fail the impossible cases.")
 
 Review comment:
   Hi @garydgregory 
   
   I received an email from github with another comment from you but cannot 
find it through the web interface so I'll repeat for clarity the comment was 
"I'm all for making Base32 and Base64 carry the same option."
   
   This PR already has the public setStrictDecoding(boolean) for Base32 and 
Base64. The unresolved is whether the property should be added to BCodec as 
well. At the moment this test is ignored because BCodec cannot be made strict. 
So either:
   
   1. Drop the test and BCodec is always lenient
   2. Make BCodec always strict (this test will pass)
   3. Make BCodec have configurable strict/lenient decoding
   
   My option would be number 3.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [commons-codec] garydgregory commented on a change in pull request #35: [CODEC-280] Added strict decoding property to BaseNCodec.

[GitBox]

garydgregory commented on a change in pull request #35: [CODEC-280] Added 
strict decoding property to BaseNCodec.
URL: https://github.com/apache/commons-codec/pull/35#discussion_r370959063
 
 

 ##
 File path: src/test/java/org/apache/commons/codec/net/BCodecTest.java
 ##
 @@ -157,6 +158,7 @@ public void testDecodeObjects() throws Exception {
 }
 
 @Test
+@Ignore("CODEC-280: Disabled strict decoding by default. The BCodec uses 
the default so this test does not fail the impossible cases.")
 
 Review comment:
   I'm all for making Base32 and Base64 carry the same option.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [commons-codec] garydgregory commented on a change in pull request #35: [CODEC-280] Added strict decoding property to BaseNCodec.

[GitBox]

garydgregory commented on a change in pull request #35: [CODEC-280] Added 
strict decoding property to BaseNCodec.
URL: https://github.com/apache/commons-codec/pull/35#discussion_r370959063
 
 

 ##
 File path: src/test/java/org/apache/commons/codec/net/BCodecTest.java
 ##
 @@ -157,6 +158,7 @@ public void testDecodeObjects() throws Exception {
 }
 
 @Test
+@Ignore("CODEC-280: Disabled strict decoding by default. The BCodec uses 
the default so this test does not fail the impossible cases.")
 
 Review comment:
   I'm all for making Base32 and Base64 carry the same option.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Commented] (COMPRESS-502) Allow to disable closing files in the finalizer of ZipFile

[Dominik Stadler (Jira)]

[ 
https://issues.apache.org/jira/browse/COMPRESS-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023640#comment-17023640
 ] 

Dominik Stadler commented on COMPRESS-502:
--

We would like to use another tool to better report the actual location of where 
the file was opened as the single output to stderr can be too less to find the 
location of allocation in a fairly sized application.

So I mostly would like to not have "close()" be run as part of the finalizer.

The output to stderr is another shortcoming which might be unwanted for others.

A simple solution as far as I see would be to make it pluggable via a simple 
static setter and use the current System.err.println as default.

 

E.g. something like the following (did not compile this, so might not fully 
work this way):

 

private static Function finalizeFun = (file) -> {

    if (!closed) {
    System.err.println("Cleaning up unclosed ZipFile for archive " + 
archiveName);
            file.close();
    }
 };

public static void setFinalizer(Function fun) {

    finalizeFun = fun;

}

protected void finalize() throws Throwable {

    try {

        finalizeRunnable->apply(this);

    } finally {
   super.finalize();
    }

}

 

> Allow to disable closing files in the finalizer of ZipFile
> --
>
> Key: COMPRESS-502
> URL: https://issues.apache.org/jira/browse/COMPRESS-502
> Project: Commons Compress
>  Issue Type: Improvement
>  Components: Compressors
>Affects Versions: 1.19
>Reporter: Dominik Stadler
>Priority: Major
>
> Apache POI uses commons-compress for handling ZipFiles. We found that it 
> sometimes does some auto-close magic in the finalizer of the ZipFile class 
> with printing out to stderr, see 
> https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=blob;f=src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java;h=23194560dace91d8052626f3bdc8f765f9c46d7e;hb=HEAD#l652.
>  
> This has some shortcomings:
>  * It prints to stderr which many large-scale applications try to avoid by 
> using some logging framework, thus this output might "vanish" unseen in some 
> installations or might cause unexpected side-effects
>  * It prevents us from using tools for checking file leaks, e.g. we use 
> [https://github.com/kohsuke/file-leak-detector/] heavily for analyzing 
> test-runs for leaked file-handles, but commons-compress prevents this because 
> it "hides" the unclosed file from this functionality
>  * The behavior of automatic closing and reporting the problem is 
> non-reproducible because it depends on finalizer/garbage-collection and thus 
> any re-runs or unit-tests usually do not show the same behavior
>  
> There are some fairly simple options to improve this:
>  * Allow to disable this functionality via configuration/system-property/...
>  * Make this "pluggable" so a logging-framework can be plugged-in or closing 
> can be prevented for certain runs
>  
> I can provide a simple patch if you state which approach you think would make 
> most sense here.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [commons-csv] garydgregory commented on issue #54: Add testcases for CSVRecord with get(Enum) and toString.

[GitBox]

garydgregory commented on issue #54: Add testcases for CSVRecord with get(Enum) 
and toString.
URL: https://github.com/apache/commons-csv/pull/54#issuecomment-578429589
 
 
   @dota17 
   May you please resolve conflicts? I'm afraid master's diverged enough since 
you created this PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [commons-codec] aherbert commented on a change in pull request #35: [CODEC-280] Added strict decoding property to BaseNCodec.

[GitBox]

aherbert commented on a change in pull request #35: [CODEC-280] Added strict 
decoding property to BaseNCodec.
URL: https://github.com/apache/commons-codec/pull/35#discussion_r370947730
 
 

 ##
 File path: src/test/java/org/apache/commons/codec/net/BCodecTest.java
 ##
 @@ -157,6 +158,7 @@ public void testDecodeObjects() throws Exception {
 }
 
 @Test
+@Ignore("CODEC-280: Disabled strict decoding by default. The BCodec uses 
the default so this test does not fail the impossible cases.")
 
 Review comment:
   The new behaviour is lenient. So that would be a behaviour change for BCodec 
and make this test redundant.
   
   I think that the new strict decoding property can be added here too so users 
have an option.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [commons-codec] garydgregory commented on a change in pull request #35: [CODEC-280] Added strict decoding property to BaseNCodec.

[GitBox]

garydgregory commented on a change in pull request #35: [CODEC-280] Added 
strict decoding property to BaseNCodec.
URL: https://github.com/apache/commons-codec/pull/35#discussion_r370946325
 
 

 ##
 File path: src/test/java/org/apache/commons/codec/net/BCodecTest.java
 ##
 @@ -157,6 +158,7 @@ public void testDecodeObjects() throws Exception {
 }
 
 @Test
+@Ignore("CODEC-280: Disabled strict decoding by default. The BCodec uses 
the default so this test does not fail the impossible cases.")
 
 Review comment:
   @aherbert 
   I think this test should be rewritten to match the new behavior, not 
`@Ignore`'d.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (COMPRESS-493) Update optional library com.github.luben:zstd-jni from 1.4.0-1 to 1.4.4-7

[Gary D. Gregory (Jira)]

 [ 
https://issues.apache.org/jira/browse/COMPRESS-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary D. Gregory updated COMPRESS-493:
-
Summary: Update optional library com.github.luben:zstd-jni from 1.4.0-1 to 
1.4.4-7  (was: Update optional library com.github.luben:zstd-jni from 1.4.0-1 
to 1.4.3-1)

> Update optional library com.github.luben:zstd-jni from 1.4.0-1 to 1.4.4-7
> -
>
> Key: COMPRESS-493
> URL: https://issues.apache.org/jira/browse/COMPRESS-493
> Project: Commons Compress
>  Issue Type: Improvement
>Reporter: Gary D. Gregory
>Priority: Major
> Fix For: 1.20
>
>
> Update Optional library {{com.github.luben:zstd-jni}} from 1.4.0-1 to 1.4.3-1.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Closed] (COMPRESS-493) Update optional library com.github.luben:zstd-jni from 1.4.0-1 to 1.4.4-7

[Gary D. Gregory (Jira)]

 [ 
https://issues.apache.org/jira/browse/COMPRESS-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary D. Gregory closed COMPRESS-493.

Resolution: Fixed

> Update optional library com.github.luben:zstd-jni from 1.4.0-1 to 1.4.4-7
> -
>
> Key: COMPRESS-493
> URL: https://issues.apache.org/jira/browse/COMPRESS-493
> Project: Commons Compress
>  Issue Type: Improvement
>Reporter: Gary D. Gregory
>Priority: Major
> Fix For: 1.20
>
>
> Update Optional library {{com.github.luben:zstd-jni}} from 1.4.0-1 to 1.4.4-7.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (COMPRESS-493) Update optional library com.github.luben:zstd-jni from 1.4.0-1 to 1.4.4-7

[Gary D. Gregory (Jira)]

 [ 
https://issues.apache.org/jira/browse/COMPRESS-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary D. Gregory updated COMPRESS-493:
-
Description: Update Optional library {{com.github.luben:zstd-jni}} from 
1.4.0-1 to 1.4.4-7.  (was: Update Optional library 
{{com.github.luben:zstd-jni}} from 1.4.0-1 to 1.4.3-1.)

> Update optional library com.github.luben:zstd-jni from 1.4.0-1 to 1.4.4-7
> -
>
> Key: COMPRESS-493
> URL: https://issues.apache.org/jira/browse/COMPRESS-493
> Project: Commons Compress
>  Issue Type: Improvement
>Reporter: Gary D. Gregory
>Priority: Major
> Fix For: 1.20
>
>
> Update Optional library {{com.github.luben:zstd-jni}} from 1.4.0-1 to 1.4.4-7.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Reopened] (COMPRESS-493) Update optional library com.github.luben:zstd-jni from 1.4.0-1 to 1.4.3-1

[Gary D. Gregory (Jira)]

 [ 
https://issues.apache.org/jira/browse/COMPRESS-493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary D. Gregory reopened COMPRESS-493:
--

> Update optional library com.github.luben:zstd-jni from 1.4.0-1 to 1.4.3-1
> -
>
> Key: COMPRESS-493
> URL: https://issues.apache.org/jira/browse/COMPRESS-493
> Project: Commons Compress
>  Issue Type: Improvement
>Reporter: Gary D. Gregory
>Priority: Major
> Fix For: 1.20
>
>
> Update Optional library {{com.github.luben:zstd-jni}} from 1.4.0-1 to 1.4.3-1.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (COMPRESS-502) Allow to disable closing files in the finalizer of ZipFile

[Stefan Bodewig (Jira)]

[ 
https://issues.apache.org/jira/browse/COMPRESS-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023583#comment-17023583
 ] 

Stefan Bodewig commented on COMPRESS-502:
-

Hmm, if the ZipFile instance is properly closed then there is no message at 
all. If people want to disable the feature because they know they always close 
the file then they wouldn't need to disable it because the message never 
appears :)

> Allow to disable closing files in the finalizer of ZipFile
> --
>
> Key: COMPRESS-502
> URL: https://issues.apache.org/jira/browse/COMPRESS-502
> Project: Commons Compress
>  Issue Type: Improvement
>  Components: Compressors
>Affects Versions: 1.19
>Reporter: Dominik Stadler
>Priority: Major
>
> Apache POI uses commons-compress for handling ZipFiles. We found that it 
> sometimes does some auto-close magic in the finalizer of the ZipFile class 
> with printing out to stderr, see 
> https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=blob;f=src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java;h=23194560dace91d8052626f3bdc8f765f9c46d7e;hb=HEAD#l652.
>  
> This has some shortcomings:
>  * It prints to stderr which many large-scale applications try to avoid by 
> using some logging framework, thus this output might "vanish" unseen in some 
> installations or might cause unexpected side-effects
>  * It prevents us from using tools for checking file leaks, e.g. we use 
> [https://github.com/kohsuke/file-leak-detector/] heavily for analyzing 
> test-runs for leaked file-handles, but commons-compress prevents this because 
> it "hides" the unclosed file from this functionality
>  * The behavior of automatic closing and reporting the problem is 
> non-reproducible because it depends on finalizer/garbage-collection and thus 
> any re-runs or unit-tests usually do not show the same behavior
>  
> There are some fairly simple options to improve this:
>  * Allow to disable this functionality via configuration/system-property/...
>  * Make this "pluggable" so a logging-framework can be plugged-in or closing 
> can be prevented for certain runs
>  
> I can provide a simple patch if you state which approach you think would make 
> most sense here.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (COMPRESS-502) Allow to disable closing files in the finalizer of ZipFile

[Stefan Bodewig (Jira)]

[ 
https://issues.apache.org/jira/browse/COMPRESS-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023582#comment-17023582
 ] 

Stefan Bodewig commented on COMPRESS-502:
-

I'm afraid the finalizer has there for a very long time and we can't simply 
remove it.

Plugging in a logging framework would address the message going to System.err 
but not address the two other shortcomings you mention.

In the case of Tika you probably want to make sure to never run the finalizer 
which probably rules out a system property. This would leave us with yet 
another constructor argument.

> Allow to disable closing files in the finalizer of ZipFile
> --
>
> Key: COMPRESS-502
> URL: https://issues.apache.org/jira/browse/COMPRESS-502
> Project: Commons Compress
>  Issue Type: Improvement
>  Components: Compressors
>Affects Versions: 1.19
>Reporter: Dominik Stadler
>Priority: Major
>
> Apache POI uses commons-compress for handling ZipFiles. We found that it 
> sometimes does some auto-close magic in the finalizer of the ZipFile class 
> with printing out to stderr, see 
> https://gitbox.apache.org/repos/asf?p=commons-compress.git;a=blob;f=src/main/java/org/apache/commons/compress/archivers/zip/ZipFile.java;h=23194560dace91d8052626f3bdc8f765f9c46d7e;hb=HEAD#l652.
>  
> This has some shortcomings:
>  * It prints to stderr which many large-scale applications try to avoid by 
> using some logging framework, thus this output might "vanish" unseen in some 
> installations or might cause unexpected side-effects
>  * It prevents us from using tools for checking file leaks, e.g. we use 
> [https://github.com/kohsuke/file-leak-detector/] heavily for analyzing 
> test-runs for leaked file-handles, but commons-compress prevents this because 
> it "hides" the unclosed file from this functionality
>  * The behavior of automatic closing and reporting the problem is 
> non-reproducible because it depends on finalizer/garbage-collection and thus 
> any re-runs or unit-tests usually do not show the same behavior
>  
> There are some fairly simple options to improve this:
>  * Allow to disable this functionality via configuration/system-property/...
>  * Make this "pluggable" so a logging-framework can be plugged-in or closing 
> can be prevented for certain runs
>  
> I can provide a simple patch if you state which approach you think would make 
> most sense here.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (COMPRESS-124) Unable to extract a sparse entries from tar archives

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/COMPRESS-124?focusedWorklogId=377205&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377205
 ]

ASF GitHub Bot logged work on COMPRESS-124:
---

Author: ASF GitHub Bot
Created on: 25/Jan/20 14:13
Start Date: 25/Jan/20 14:13
Worklog Time Spent: 10m 
  Work Description: bodewig commented on pull request #89: COMPRESS-124 : 
minor cleanups
URL: https://github.com/apache/commons-compress/pull/89
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 377205)
Time Spent: 6h 20m  (was: 6h 10m)

> Unable to extract a sparse entries from tar archives
> 
>
> Key: COMPRESS-124
> URL: https://issues.apache.org/jira/browse/COMPRESS-124
> Project: Commons Compress
>  Issue Type: New Feature
>  Components: Archivers
>Affects Versions: 1.1, 1.2
> Environment: Platform independent. However, I'm currently using 
> Window 7 Enterprise.
>Reporter: Patrick Dreyer
>Priority: Major
>  Labels: tar
> Fix For: 1.20
>
> Attachments: gnuSparseFile.patch
>
>  Time Spent: 6h 20m
>  Remaining Estimate: 0h
>
> Good news first: I already have the patch ready for that.
> I got several TAR files which I could not extract with any of the existing 
> Java implementations, but I could extract all those TAR files successfully 
> with GNU tar.
> It turned out that all the failing TAR files contained so called sparse 
> files. Investigating the source code of all existing Java TAR implementations 
> showed me that none of them even recognizes the existence of GNU sparse 
> entries.
> Actually, I don't need to process one of the contained sparse files and I'm 
> happy if I'm at least able to correctly untar all the non-sparsed files. 
> Thus, it would be sufficient recognizing sparse files without the need to 
> correctly un-sparse them while extracting. As long as all non-sparsed files 
> get extracted correctly, I'm fine.
> The TAR files in question have all been VMware Diagnostic File bundles.
> See 
> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=653
>  to know how to get them.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (COMPRESS-124) Unable to extract a sparse entries from tar archives

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/COMPRESS-124?focusedWorklogId=377204&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377204
 ]

ASF GitHub Bot logged work on COMPRESS-124:
---

Author: ASF GitHub Bot
Created on: 25/Jan/20 14:13
Start Date: 25/Jan/20 14:13
Worklog Time Spent: 10m 
  Work Description: bodewig commented on issue #89: COMPRESS-124 : minor 
cleanups
URL: https://github.com/apache/commons-compress/pull/89#issuecomment-578409525
 
 
   Somehow I managed to overlook this PR, sorry @PeterAlfreadLee . I've 
committed my own version which looks remarkable similar to yours with 
https://github.com/apache/commons-compress/commit/80657d31997ed64aeef57376402985258430a8f9
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 377204)
Time Spent: 6h 10m  (was: 6h)

> Unable to extract a sparse entries from tar archives
> 
>
> Key: COMPRESS-124
> URL: https://issues.apache.org/jira/browse/COMPRESS-124
> Project: Commons Compress
>  Issue Type: New Feature
>  Components: Archivers
>Affects Versions: 1.1, 1.2
> Environment: Platform independent. However, I'm currently using 
> Window 7 Enterprise.
>Reporter: Patrick Dreyer
>Priority: Major
>  Labels: tar
> Fix For: 1.20
>
> Attachments: gnuSparseFile.patch
>
>  Time Spent: 6h 10m
>  Remaining Estimate: 0h
>
> Good news first: I already have the patch ready for that.
> I got several TAR files which I could not extract with any of the existing 
> Java implementations, but I could extract all those TAR files successfully 
> with GNU tar.
> It turned out that all the failing TAR files contained so called sparse 
> files. Investigating the source code of all existing Java TAR implementations 
> showed me that none of them even recognizes the existence of GNU sparse 
> entries.
> Actually, I don't need to process one of the contained sparse files and I'm 
> happy if I'm at least able to correctly untar all the non-sparsed files. 
> Thus, it would be sufficient recognizing sparse files without the need to 
> correctly un-sparse them while extracting. As long as all non-sparsed files 
> get extracted correctly, I'm fine.
> The TAR files in question have all been VMware Diagnostic File bundles.
> See 
> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=653
>  to know how to get them.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [commons-compress] bodewig commented on issue #89: COMPRESS-124 : minor cleanups

[GitBox]

bodewig commented on issue #89: COMPRESS-124 : minor cleanups
URL: https://github.com/apache/commons-compress/pull/89#issuecomment-578409525
 
 
   Somehow I managed to overlook this PR, sorry @PeterAlfreadLee . I've 
committed my own version which looks remarkable similar to yours with 
https://github.com/apache/commons-compress/commit/80657d31997ed64aeef57376402985258430a8f9


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [commons-compress] bodewig closed pull request #89: COMPRESS-124 : minor cleanups

[GitBox]

bodewig closed pull request #89: COMPRESS-124 : minor cleanups
URL: https://github.com/apache/commons-compress/pull/89
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services