[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676792#comment-17676792
]
Gary D. Gregory commented on VFS-683:
-
Except that there are failures in builds that seem random but must be related.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Assignee: Gary D. Gregory
>Priority: Major
> Attachments: Main.java
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676766#comment-17676766
]
Ivan Bella commented on VFS-683:
I just submitted a PR that fixes this issue. Using the VFSClassLoader is not
the only way to demonstrate this particular issue. I think my solution is
relatively simple and will alleviate a good set of thread safety issues.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Assignee: Gary D. Gregory
>Priority: Major
> Attachments: Main.java
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17600074#comment-17600074
]
Gary D. Gregory commented on VFS-683:
-
I don't think there is an expectation of failure, but I do not think
multi-threading was baked in the design and coding initially. While we have
changed the code in an ad-hoc manner here and there over the years to avoid
multi-threading issues, it is still the case unless something is actually
documented as thread-safe and/or immutable, you should not assume it is safe to
use in a multi-threading context.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Assignee: Gary D. Gregory
>Priority: Major
> Attachments: Main.java
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577602#comment-17577602
]
Dave MacDonald commented on VFS-683:
I created a
[branch|https://github.com/davem62210/commons-vfs/compare/threads-and-classloading]
with a new test case to show the failure.
Are you saying this case is expected to fail? If so that is surprising.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Assignee: Gary D. Gregory
>Priority: Major
> Attachments: Main.java
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575401#comment-17575401
]
Gary D. Gregory commented on VFS-683:
-
It might be worth for you to create a PR so we can all see if the tests would
pass at the very least. A PR with a failing test would be good on its own so
anyone can reproduce the issue simply.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Assignee: Gary D. Gregory
>Priority: Major
> Attachments: Main.java
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17575316#comment-17575316
]
Dave MacDonald commented on VFS-683:
Is there any consensus on what to do about the concurrency problem with
{{{}org.apache.commons.vfs2.impl.VFSClassLoader{}}}?
Removing implementations of ZipFileObject#doAttach and ZipFileObject#doDetach
prevents the error indicated in the issue description, but I'm not sure that is
a reasonable long-term fix.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Assignee: Gary D. Gregory
>Priority: Major
> Attachments: Main.java
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699602#comment-16699602
]
Daryl Odnert commented on VFS-683:
--
I have logged VFS-684 requesting that the documentation should be fixed. I'm
reluctant to start a discussion on the mailing list about the issue as I'm not
currently a subscriber and don't want to subscribe just for the purpose of
raising this issue. If you think it's the right thing to do, we can close this
issue as a duplicate of VFS-684.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
> Attachments: Main.java
>
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699541#comment-16699541
]
Otto Fowler commented on VFS-683:
-
>From what I can see in past issues, this is not a defect. That is why they
>are not documented as thread safe or immutable. You can look at VFS-253 as
>well, which I just found.
I would suggest creating a Jira to explicitly state in documentation what is
and is not thread safe.
You may also want to start a [DISCUSS][VFS] thread on the commons dev mailing
list about the issue.
I am sorry if I have frustrated you, basically working through the issue to get
back to where you started.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Assignee: Otto Fowler
>Priority: Major
> Attachments: Main.java
>
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16699478#comment-16699478
]
Daryl Odnert commented on VFS-683:
--
[~ottobackwards] - Yes, I know there are a number of different approaches to
work around the problem. I logged this bug report as a request to either fix
the software (if there is agreement that there is a defect) or fix the
documentation if my multi-threaded application is using VFS incorrectly.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Assignee: Otto Fowler
>Priority: Major
> Attachments: Main.java
>
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696193#comment-16696193
]
Otto Fowler commented on VFS-683:
-
For example -> your code works, with the following modifications:
{code:java}
package org.apache.ottobackwards;
import org.apache.commons.vfs2.FileObject;
import org.apache.commons.vfs2.FileSystemManager;
import org.apache.commons.vfs2.FileType;
import org.apache.commons.vfs2.FileUtil;
//import org.apache.commons.vfs2.VFS;
import org.apache.commons.vfs2.impl.DefaultFileSystemManager;
import org.apache.commons.vfs2.impl.StandardFileSystemManager;
import org.apache.commons.vfs2.provider.tar.TarFileProvider;
public class Main {
/**
* The first parameter (args[0]) should be an directory within an archive
file. For example:
* tgz:file://Users/daryl/zookeeper-3.4.13.tgz!zookeeper-3.4.13/docs
*/
public static void main(String[] args) {
try {
System.out.println("File selected is " + args[0]);
Thread[] threads = new Thread[2];
for (int i = 0; i < threads.length; i++) {
threads[i] = new Thread(new FileObjectReadingThread(args[0]),
String.format("THREAD--(%d)-->",i));
threads[i].start();
}
for (int i = 0; i < threads.length; i++) {
threads[i].join(0);
}
} catch (Exception ex) {
System.out.println("Main caught " + ex.getClass().getName() + ": " +
ex.getMessage());
ex.printStackTrace(System.out);
}
}
static class FileObjectReadingThread implements Runnable {
String filePath;
FileObjectReadingThread(String pathname) {
filePath = pathname;
}
@Override
public void run() {
try {
FileSystemManager fileSystemManager = createFileSystemManager();
FileObject fileObject = fileSystemManager.resolveFile(filePath);
System.out.println("Thread " + Thread.currentThread().getName() + " " +
this + " running with FileObject " + System.identityHashCode(fileObject));
FileObject[] children = fileObject.getChildren();
for (FileObject child : children) {
if (child.getType() == FileType.FILE && child.isReadable()) {
System.out.println("Thread " + Thread.currentThread().getName() + "
" + this + " is reading " + child);
FileUtil.getContent(child);
}
Thread.sleep(1); // force a yield to other threads
}
} catch (Exception ex) {
System.out.println("Thread " + Thread.currentThread().getName() + " " +
this + " caught " + ex.getClass().getName() + ": " + ex.getMessage());
ex.printStackTrace(System.out);
}
}
private FileSystemManager createFileSystemManager() throws Exception {
DefaultFileSystemManager fileSystemManager = new
StandardFileSystemManager();
fileSystemManager.init();
return fileSystemManager;
}
}
}
{code}
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
> Attachments: Main.java
>
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
>
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696194#comment-16696194
]
Otto Fowler commented on VFS-683:
-
The things you are dealing with here:
* Static FileSystemManager
* Threads
* Non-Thread Safe Content
* Cache
* Using the FileSystem abstractly ( not having to "know" the fs type )
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
> Attachments: Main.java
>
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696181#comment-16696181
]
Otto Fowler commented on VFS-683:
-
I wonder if using the static vfs default manager can be an approach. So
instead of using the static, create your own per thread.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
> Attachments: Main.java
>
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696177#comment-16696177
]
Otto Fowler commented on VFS-683:
-
Ok,
So, with your example the issue is that each time you get the tar archive input
stream, it needs to create a new stream, closing the old, so they are taking
each other out, which is because as you state the sample File object is
resolving.
In the end, the file object content and streams are not going to be thread
safe. The issue, as you thought is the cache.
Let me see what I can come up with.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
> Attachments: Main.java
>
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696134#comment-16696134
]
Otto Fowler commented on VFS-683:
-
Thanks [~daryl.odnert], I'll take a peek and see if there is anything obvious,
that is non-usage, that can be done.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
> Attachments: Main.java
>
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695425#comment-16695425
]
Daryl Odnert commented on VFS-683:
--
Here is a Java program that demonstrates a problem where if you have more than
one thread attempting to read the contents of the children of the same archive
file, you will get exceptions thrown. The exceptions this applications receives
from VFS are not identical to the {{NullPointerException}} that is listed in
the original problem description, but I think this does demonstrate that there
is either a thread-safety issue that needs to be fixed, or some documentation
that needs to be clarified.
[^Main.java]
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
> Attachments: Main.java
>
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695202#comment-16695202
]
Bernd Eckenfels commented on VFS-683:
-
Just FYI, I had similiar problems. In my case I made sure that the overlay
filesystem I was using had a different filesystem instance, so it was not
closing for other threads. But there are definitly a few concurrency problems
(and if there are not concurrency problems then there are
cleanup/excessive-clone problems). Let me see if I can find the details of my
research in that area.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695032#comment-16695032
]
Daryl Odnert commented on VFS-683:
--
All of your comments are consistent with my concern that the caching mechanism
in {{AbstractFileSystem.resolveFile}} is returning the same {{FileObject}}
instance to two different threads, which is apparently a dangerous thing to do.
I'll try to put together a small sample application that reproduces the problem
soon.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694200#comment-16694200
]
Otto Fowler commented on VFS-683:
-
As to in general... The interface for FileObject is pretty clear that there
should be only one input stream opened for any one file at any one time, at
least that is my take. It would be nice if it where more explicit from a
higher level.
You may want to send something to the commons list with a [VFS] subject.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694189#comment-16694189
]
Otto Fowler commented on VFS-683:
-
I can't speak to the intent, I'm not one of the original authors, and have only
worked with the library myself as you probably have in a couple of projects, as
well as trying to fix a couple of bugs. And I am wrong a lot ;)
Further investigation:
If you look at DefaultFileContent, it looks like we create multiple input
streams and store them per thread in thread local. And then they are closed as
such.
BUT: If this is a ZipFileObject: The java util zip library says:
{code:java}
/**
* Returns an input stream for reading the contents of the specified
* zip file entry.
*
* Closing this ZIP file will, in turn, close all input
* streams that have been returned by invocations of this method.
*
* @param entry the zip file entry
* @return the input stream for reading the contents of the specified
* zip file entry.
* @throws ZipException if a ZIP format error has occurred
* @throws IOException if an I/O error has occurred
* @throws IllegalStateException if the zip file has been closed
*/
public InputStream getInputStream(ZipEntry entry) throws IOException {
{code}
Which doesn't seem good.
Still would need to reproduce
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694097#comment-16694097
]
Daryl Odnert commented on VFS-683:
--
[~ottobackwards] - In general, are {{FileObject}} instances supposed to be
thread-safe? The documentation is unclear about this.
My colleague and I are suspicious of the caching that's happening in
{{org.apache.commons.vfs2.provider.AbstractFileSystem.resolveFile}}. [Source
code is
here|https://svn.apache.org/viewvc/commons/proper/vfs/tags/commons-vfs2-project-2.2/commons-vfs2/src/main/java/org/apache/commons/vfs2/provider/AbstractFileSystem.java?revision=1811361=markup#l303].
With caching happening here, the same {{FileObject}} instance could be
returned to multiple threads, which would lead to a problem if more than one of
those threads try to read the contents simultaneously.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693981#comment-16693981
]
Otto Fowler commented on VFS-683:
-
{code:java}
/**
* Returns an input stream for reading the file's content.
*
* There may only be a single input or output stream open for the file at any
time.
*
* @return An input stream to read the file's content from. The input stream is
buffered, so there is no need to
* wrap it in a {@code BufferedInputStream}.
* @throws FileSystemException If the file does not exist, or is being read, or
is being written, or on error
* opening the stream.
*/
InputStream getInputStream() throws FileSystemException;
{code}
Just from looking through things, It doesn't seem correct to have two threads
going through here.
The Resource.getBytes() call explicitly CLOSES the stream. So, it isn't
thread safe for two readers if there is only ever one stream from the same
object
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (VFS-683) Thread safety issue in VFSClassLoader - NullPointerException thrown
[
https://issues.apache.org/jira/browse/VFS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16693974#comment-16693974
]
Otto Fowler commented on VFS-683:
-
any kind of small sample project that can reproduce the issue would be helpful.
> Thread safety issue in VFSClassLoader - NullPointerException thrown
> ---
>
> Key: VFS-683
> URL: https://issues.apache.org/jira/browse/VFS-683
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.2
>Reporter: Daryl Odnert
>Priority: Major
>
> In my application, I have two instances of the {{VFSClassLoader}}, each of
> which is being used in a distinct thread. Both {{VFSClassLoader}} instances
> refer to the same compressed file resource described by a {{FileObject}} that
> is passed to the class loader's constructor. Intermittently, the application
> throws an exception with the stack trace shown below. So, there seems to be
> either a race condition in the code or an undocumented assumption here. If it
> is unsupported for two {{VFSClassLoader}} instances to refer to the same
> resource (file), then that assumption should be documented. But if that is
> not the case, then there is a race condition bug in the implementation.
> {noformat}
> 43789 WARN {} c.a.e.u.PreferredPathClassLoader - While loading class
> org.apache.hive.jdbc.HiveDatabaseMetaData, rethrowing unexpected
> java.lang.NullPointerException: Inflater has been closed
> java.lang.NullPointerException: Inflater has been closed
> at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
> at java.util.zip.Inflater.inflate(Inflater.java:257)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:91)
> at org.apache.commons.vfs2.FileUtil.getContent(FileUtil.java:47)
> at org.apache.commons.vfs2.impl.Resource.getBytes(Resource.java:102)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.defineClass(VFSClassLoader.java:179)
> at
> org.apache.commons.vfs2.impl.VFSClassLoader.findClass(VFSClassLoader.java:150)
> at
> com.atscale.engine.utils.PreferredPathClassLoader.findClass(PreferredPathClassLoader.scala:54)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
