[jira] [Commented] (VFS-807) LocalFile migration to NIO is not done correctly
[
https://issues.apache.org/jira/browse/VFS-807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17449396#comment-17449396
]
L commented on VFS-807:
---
[~ggregory] : yes, it can be closed.
> LocalFile migration to NIO is not done correctly
>
>
> Key: VFS-807
> URL: https://issues.apache.org/jira/browse/VFS-807
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.9.0
>Reporter: L
>Priority: Blocker
> Fix For: 2.9.1
>
> Attachments: TestJdkWriteFile.java, vfs-807.diff
>
>
> According to release notes
> ([https://archive.apache.org/dist/commons/vfs/RELEASE-NOTES.txt),] version
> 2.9.0 includes the following change:
> Replace construction of FileInputStream and FileOutputStream objects with
> Files NIO APIs. #164. Thanks to Arturo Bernal.
> Strangely, it was not completed: LocalFlie.java has the following now, even
> in master:
> {code:java}
> /**
> * Creates an input stream to read the content from.
> */
> @Override
> protected InputStream doGetInputStream(final int bufferSize) throws
> Exception {
> return new FileInputStream(file);
> }
> {code}
>
> But what is worse, doGetOutputStream is *completely* broken:
> {code:java}
> /**
> * Creates an output stream to write the file content to.
> */
> @Override
> protected OutputStream doGetOutputStream(final boolean bAppend) throws
> Exception {
> return Files.newOutputStream(file.toPath(), bAppend ?
> StandardOpenOption.APPEND : StandardOpenOption.CREATE);
> } {code}
>
> This can be demonstrated with a small java program, see attachment. The
> program just tries to write to a local file (into the same directory), using
> different ways to do it:
> # Java IO, how it was done before VFS 2.9.0
> # Java NIO the way VFS 2.9.0 does it
> # Java NIO the way it must be done
> What is important, the program also verifies the result.
> VFS 2.9.0 fails in 2 cases:
> # When the file does not exist and append must be performed:
> java.nio.file.NoSuchFileException is thrown in this case.
> # When the file exists and no append must be performed: the file is not
> truncated, so whatever is written overwrites the beginning of the file,
> keeping the rest.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (VFS-807) LocalFile migration to NIO is not done correctly
[
https://issues.apache.org/jira/browse/VFS-807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394698#comment-17394698
]
L commented on VFS-807:
---
[~ggregory]
I am using Windows 10 and jdk 8 as well.
The thing is: you did not correctly pushed my test changes, namely,
ProviderWriteAppendTests.java:
Mine code:
{code:java}
try (OutputStream os = file.getContent().getOutputStream(true)) {
os.write(content1.getBytes(StandardCharsets.UTF_8));
}
{code}
Code in master:
{code:java}
try (FileContent fileContent = file.getContent(); OutputStream os =
fileContent.getOutputStream()) {
os.write(content1.getBytes(StandardCharsets.UTF_8));
}
{code}
Your code does not perform "append". It must be
fileContent.getOutputStream(*true*)
> LocalFile migration to NIO is not done correctly
>
>
> Key: VFS-807
> URL: https://issues.apache.org/jira/browse/VFS-807
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.9.0
>Reporter: L
>Priority: Blocker
> Fix For: 2.9.1
>
> Attachments: TestJdkWriteFile.java, vfs-807.diff
>
>
> According to release notes
> ([https://archive.apache.org/dist/commons/vfs/RELEASE-NOTES.txt),] version
> 2.9.0 includes the following change:
> Replace construction of FileInputStream and FileOutputStream objects with
> Files NIO APIs. #164. Thanks to Arturo Bernal.
> Strangely, it was not completed: LocalFlie.java has the following now, even
> in master:
> {code:java}
> /**
> * Creates an input stream to read the content from.
> */
> @Override
> protected InputStream doGetInputStream(final int bufferSize) throws
> Exception {
> return new FileInputStream(file);
> }
> {code}
>
> But what is worse, doGetOutputStream is *completely* broken:
> {code:java}
> /**
> * Creates an output stream to write the file content to.
> */
> @Override
> protected OutputStream doGetOutputStream(final boolean bAppend) throws
> Exception {
> return Files.newOutputStream(file.toPath(), bAppend ?
> StandardOpenOption.APPEND : StandardOpenOption.CREATE);
> } {code}
>
> This can be demonstrated with a small java program, see attachment. The
> program just tries to write to a local file (into the same directory), using
> different ways to do it:
> # Java IO, how it was done before VFS 2.9.0
> # Java NIO the way VFS 2.9.0 does it
> # Java NIO the way it must be done
> What is important, the program also verifies the result.
> VFS 2.9.0 fails in 2 cases:
> # When the file does not exist and append must be performed:
> java.nio.file.NoSuchFileException is thrown in this case.
> # When the file exists and no append must be performed: the file is not
> truncated, so whatever is written overwrites the beginning of the file,
> keeping the rest.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (VFS-807) LocalFile migration to NIO is not done correctly
[
https://issues.apache.org/jira/browse/VFS-807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17394536#comment-17394536
]
L commented on VFS-807:
---
[~ggregory]: thanks for pushing the tests.
As for "the tests pass without the change to {{LocalFile}} ": of course they
pass now since you *have changed* LocalFile (the same fix as Commons IO)
finally fixing the problem. My comment with diff file was posted several days
ago when LocalFile in master was still buggy.
> LocalFile migration to NIO is not done correctly
>
>
> Key: VFS-807
> URL: https://issues.apache.org/jira/browse/VFS-807
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.9.0
>Reporter: L
>Priority: Blocker
> Fix For: 2.9.1
>
> Attachments: TestJdkWriteFile.java, vfs-807.diff
>
>
> According to release notes
> ([https://archive.apache.org/dist/commons/vfs/RELEASE-NOTES.txt),] version
> 2.9.0 includes the following change:
> Replace construction of FileInputStream and FileOutputStream objects with
> Files NIO APIs. #164. Thanks to Arturo Bernal.
> Strangely, it was not completed: LocalFlie.java has the following now, even
> in master:
> {code:java}
> /**
> * Creates an input stream to read the content from.
> */
> @Override
> protected InputStream doGetInputStream(final int bufferSize) throws
> Exception {
> return new FileInputStream(file);
> }
> {code}
>
> But what is worse, doGetOutputStream is *completely* broken:
> {code:java}
> /**
> * Creates an output stream to write the file content to.
> */
> @Override
> protected OutputStream doGetOutputStream(final boolean bAppend) throws
> Exception {
> return Files.newOutputStream(file.toPath(), bAppend ?
> StandardOpenOption.APPEND : StandardOpenOption.CREATE);
> } {code}
>
> This can be demonstrated with a small java program, see attachment. The
> program just tries to write to a local file (into the same directory), using
> different ways to do it:
> # Java IO, how it was done before VFS 2.9.0
> # Java NIO the way VFS 2.9.0 does it
> # Java NIO the way it must be done
> What is important, the program also verifies the result.
> VFS 2.9.0 fails in 2 cases:
> # When the file does not exist and append must be performed:
> java.nio.file.NoSuchFileException is thrown in this case.
> # When the file exists and no append must be performed: the file is not
> truncated, so whatever is written overwrites the beginning of the file,
> keeping the rest.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (VFS-807) LocalFile migration to NIO is not done correctly
[
https://issues.apache.org/jira/browse/VFS-807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17391433#comment-17391433
]
L commented on VFS-807:
---
I have added some tests verifying the bug. I did not succeed creating a PR.
The attached file (vfs-807.diff) contains the test changes and the correct fix
for this issue.
# ProviderWriteTests.testOverwriteContent() fails with 2.9.0 and works in the
latest master.
# ProviderWriteAppendTests.testAppendToNonExsiting() fails with 2.9.0 and in
the latest master.
# They both succeed with the changes to LocalFile done in the patch.
> LocalFile migration to NIO is not done correctly
>
>
> Key: VFS-807
> URL: https://issues.apache.org/jira/browse/VFS-807
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.9.0
>Reporter: L
>Priority: Blocker
> Fix For: 2.9.1
>
> Attachments: TestJdkWriteFile.java, vfs-807.diff
>
>
> According to release notes
> ([https://archive.apache.org/dist/commons/vfs/RELEASE-NOTES.txt),] version
> 2.9.0 includes the following change:
> Replace construction of FileInputStream and FileOutputStream objects with
> Files NIO APIs. #164. Thanks to Arturo Bernal.
> Strangely, it was not completed: LocalFlie.java has the following now, even
> in master:
> {code:java}
> /**
> * Creates an input stream to read the content from.
> */
> @Override
> protected InputStream doGetInputStream(final int bufferSize) throws
> Exception {
> return new FileInputStream(file);
> }
> {code}
>
> But what is worse, doGetOutputStream is *completely* broken:
> {code:java}
> /**
> * Creates an output stream to write the file content to.
> */
> @Override
> protected OutputStream doGetOutputStream(final boolean bAppend) throws
> Exception {
> return Files.newOutputStream(file.toPath(), bAppend ?
> StandardOpenOption.APPEND : StandardOpenOption.CREATE);
> } {code}
>
> This can be demonstrated with a small java program, see attachment. The
> program just tries to write to a local file (into the same directory), using
> different ways to do it:
> # Java IO, how it was done before VFS 2.9.0
> # Java NIO the way VFS 2.9.0 does it
> # Java NIO the way it must be done
> What is important, the program also verifies the result.
> VFS 2.9.0 fails in 2 cases:
> # When the file does not exist and append must be performed:
> java.nio.file.NoSuchFileException is thrown in this case.
> # When the file exists and no append must be performed: the file is not
> truncated, so whatever is written overwrites the beginning of the file,
> keeping the rest.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (VFS-807) LocalFile migration to NIO is not done correctly
[
https://issues.apache.org/jira/browse/VFS-807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L updated VFS-807:
--
Attachment: vfs-807.diff
> LocalFile migration to NIO is not done correctly
>
>
> Key: VFS-807
> URL: https://issues.apache.org/jira/browse/VFS-807
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.9.0
>Reporter: L
>Priority: Blocker
> Fix For: 2.9.1
>
> Attachments: TestJdkWriteFile.java, vfs-807.diff
>
>
> According to release notes
> ([https://archive.apache.org/dist/commons/vfs/RELEASE-NOTES.txt),] version
> 2.9.0 includes the following change:
> Replace construction of FileInputStream and FileOutputStream objects with
> Files NIO APIs. #164. Thanks to Arturo Bernal.
> Strangely, it was not completed: LocalFlie.java has the following now, even
> in master:
> {code:java}
> /**
> * Creates an input stream to read the content from.
> */
> @Override
> protected InputStream doGetInputStream(final int bufferSize) throws
> Exception {
> return new FileInputStream(file);
> }
> {code}
>
> But what is worse, doGetOutputStream is *completely* broken:
> {code:java}
> /**
> * Creates an output stream to write the file content to.
> */
> @Override
> protected OutputStream doGetOutputStream(final boolean bAppend) throws
> Exception {
> return Files.newOutputStream(file.toPath(), bAppend ?
> StandardOpenOption.APPEND : StandardOpenOption.CREATE);
> } {code}
>
> This can be demonstrated with a small java program, see attachment. The
> program just tries to write to a local file (into the same directory), using
> different ways to do it:
> # Java IO, how it was done before VFS 2.9.0
> # Java NIO the way VFS 2.9.0 does it
> # Java NIO the way it must be done
> What is important, the program also verifies the result.
> VFS 2.9.0 fails in 2 cases:
> # When the file does not exist and append must be performed:
> java.nio.file.NoSuchFileException is thrown in this case.
> # When the file exists and no append must be performed: the file is not
> truncated, so whatever is written overwrites the beginning of the file,
> keeping the rest.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (VFS-807) LocalFile migration to NIO is not done correctly
[
https://issues.apache.org/jira/browse/VFS-807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L reopened VFS-807:
---
Reopening: the fix is not complete.
It fixes only part of the problem: overwriting the existing file.
The second part: appending to a non-existing file still fails
> LocalFile migration to NIO is not done correctly
>
>
> Key: VFS-807
> URL: https://issues.apache.org/jira/browse/VFS-807
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.9.0
>Reporter: L
>Priority: Blocker
> Fix For: 2.9.1
>
> Attachments: TestJdkWriteFile.java
>
>
> According to release notes
> ([https://archive.apache.org/dist/commons/vfs/RELEASE-NOTES.txt),] version
> 2.9.0 includes the following change:
> Replace construction of FileInputStream and FileOutputStream objects with
> Files NIO APIs. #164. Thanks to Arturo Bernal.
> Strangely, it was not completed: LocalFlie.java has the following now, even
> in master:
> {code:java}
> /**
> * Creates an input stream to read the content from.
> */
> @Override
> protected InputStream doGetInputStream(final int bufferSize) throws
> Exception {
> return new FileInputStream(file);
> }
> {code}
>
> But what is worse, doGetOutputStream is *completely* broken:
> {code:java}
> /**
> * Creates an output stream to write the file content to.
> */
> @Override
> protected OutputStream doGetOutputStream(final boolean bAppend) throws
> Exception {
> return Files.newOutputStream(file.toPath(), bAppend ?
> StandardOpenOption.APPEND : StandardOpenOption.CREATE);
> } {code}
>
> This can be demonstrated with a small java program, see attachment. The
> program just tries to write to a local file (into the same directory), using
> different ways to do it:
> # Java IO, how it was done before VFS 2.9.0
> # Java NIO the way VFS 2.9.0 does it
> # Java NIO the way it must be done
> What is important, the program also verifies the result.
> VFS 2.9.0 fails in 2 cases:
> # When the file does not exist and append must be performed:
> java.nio.file.NoSuchFileException is thrown in this case.
> # When the file exists and no append must be performed: the file is not
> truncated, so whatever is written overwrites the beginning of the file,
> keeping the rest.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (VFS-807) LocalFile migration to NIO is not done correctly
[
https://issues.apache.org/jira/browse/VFS-807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L updated VFS-807:
--
Priority: Blocker (was: Critical)
> LocalFile migration to NIO is not done correctly
>
>
> Key: VFS-807
> URL: https://issues.apache.org/jira/browse/VFS-807
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.9.0
>Reporter: L
>Priority: Blocker
> Attachments: TestJdkWriteFile.java
>
>
> According to release notes
> ([https://archive.apache.org/dist/commons/vfs/RELEASE-NOTES.txt),] version
> 2.9.0 includes the following change:
> Replace construction of FileInputStream and FileOutputStream objects with
> Files NIO APIs. #164. Thanks to Arturo Bernal.
> Strangely, it was not completed: LocalFlie.java has the following now, even
> in master:
> {code:java}
> /**
> * Creates an input stream to read the content from.
> */
> @Override
> protected InputStream doGetInputStream(final int bufferSize) throws
> Exception {
> return new FileInputStream(file);
> }
> {code}
>
> But what is worse, doGetOutputStream is *completely* broken:
> {code:java}
> /**
> * Creates an output stream to write the file content to.
> */
> @Override
> protected OutputStream doGetOutputStream(final boolean bAppend) throws
> Exception {
> return Files.newOutputStream(file.toPath(), bAppend ?
> StandardOpenOption.APPEND : StandardOpenOption.CREATE);
> } {code}
>
> This can be demonstrated with a small java program, see attachment. The
> program just tries to write to a local file (into the same directory), using
> different ways to do it:
> # Java IO, how it was done before VFS 2.9.0
> # Java NIO the way VFS 2.9.0 does it
> # Java NIO the way it must be done
> What is important, the program also verifies the result.
> VFS 2.9.0 fails in 2 cases:
> # When the file does not exist and append must be performed:
> java.nio.file.NoSuchFileException is thrown in this case.
> # When the file exists and no append must be performed: the file is not
> truncated, so whatever is written overwrites the beginning of the file,
> keeping the rest.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (VFS-807) LocalFile migration to NIO is not done correctly
L created VFS-807:
-
Summary: LocalFile migration to NIO is not done correctly
Key: VFS-807
URL: https://issues.apache.org/jira/browse/VFS-807
Project: Commons VFS
Issue Type: Bug
Affects Versions: 2.9.0
Reporter: L
Attachments: TestJdkWriteFile.java
According to release notes
([https://archive.apache.org/dist/commons/vfs/RELEASE-NOTES.txt),] version
2.9.0 includes the following change:
Replace construction of FileInputStream and FileOutputStream objects with Files
NIO APIs. #164. Thanks to Arturo Bernal.
Strangely, it was not completed: LocalFlie.java has the following now, even in
master:
{code:java}
/**
* Creates an input stream to read the content from.
*/
@Override
protected InputStream doGetInputStream(final int bufferSize) throws Exception {
return new FileInputStream(file);
}
{code}
But what is worse, doGetOutputStream is *completely* broken:
{code:java}
/**
* Creates an output stream to write the file content to.
*/
@Override
protected OutputStream doGetOutputStream(final boolean bAppend) throws
Exception {
return Files.newOutputStream(file.toPath(), bAppend ?
StandardOpenOption.APPEND : StandardOpenOption.CREATE);
} {code}
This can be demonstrated with a small java program, see attachment. The program
just tries to write to a local file (into the same directory), using different
ways to do it:
# Java IO, how it was done before VFS 2.9.0
# Java NIO the way VFS 2.9.0 does it
# Java NIO the way it must be done
What is important, the program also verifies the result.
VFS 2.9.0 fails in 2 cases:
# When the file does not exist and append must be performed:
java.nio.file.NoSuchFileException is thrown in this case.
# When the file exists and no append must be performed: the file is not
truncated, so whatever is written overwrites the beginning of the file, keeping
the rest.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (CODEC-289) Base32/64Input/OutputStream to allow optional strict/lenient decoding
[
https://issues.apache.org/jira/browse/CODEC-289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121448#comment-17121448
]
venkata ramana reddy L commented on CODEC-289:
--
[~aherbert] - When is the release of 1.15 planned?
> Base32/64Input/OutputStream to allow optional strict/lenient decoding
> -
>
> Key: CODEC-289
> URL: https://issues.apache.org/jira/browse/CODEC-289
> Project: Commons Codec
> Issue Type: Improvement
>Affects Versions: 1.14
>Reporter: Alex Herbert
>Assignee: Alex Herbert
>Priority: Major
> Fix For: 1.15
>
>
> Add the strict decoding option from the Base32/64 decoder to the
> Input/OutputStream versions.
> Requires adding a property to BaseNCodecInputStream and
> BaseNCodecOutputStream to pass through to the underlying decoder:
> {code:java}
> public void setStrictDecoding(boolean strictDecoding) {
>baseNCodec.setStrictDecoding(strictDecoding);
> }
> public boolean isStrictDecoding() {
>return baseNCodec.isStrictDecoding();
> }
> {code}
> This would allow Base32/64 Input/Output streams to use the strict decoding
> mode. Default behaviour would be lenient decoding for backward compatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (JCS-186) Lateral cache replication is not working with UDP discovery
Alex L created JCS-186: -- Summary: Lateral cache replication is not working with UDP discovery Key: JCS-186 URL: https://issues.apache.org/jira/browse/JCS-186 Project: Commons JCS Issue Type: Bug Components: TCP Lateral Cache Affects Versions: jcs-2.2 Reporter: Alex L Hello, When using UDP discovery, the replication is not working. I'm using the basic example from the website and although the instances discover each other, there's no actual replication of cache objects. This issue only occurs with UDP discovery. When specifying the TCP Servers list everything works as exepected. The issue seems to be introduced with the refactorings in this commit: [http://svn.apache.org/viewvc/commons/proper/jcs/tags/commons-jcs-2.2/commons-jcs-core/src/main/java/org/apache/commons/jcs/auxiliary/lateral/socket/tcp/LateralTCPDiscoveryListener.java?r1=1803806=1593843_format=h] As a technical note, the tcpServer of the attributes is empty when using UDP discovery, so the DiscoveredService will never be added to the noWait list. Let me know if you need more details. Thank you, Alex -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (VFS-590) SFTP moveTo operation might fail on permission checks even if the operation itself might succeed
[
https://issues.apache.org/jira/browse/VFS-590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296810#comment-16296810
]
L edited comment on VFS-590 at 12/19/17 1:51 PM:
-
I do not know how and when this issue will be fixed.
There is a more active discussion of a very similar issue in VFS-617. IMO
VFS-617 fixes VFS-590 as well.
was (Author: klv_m72):
I do not know how and when this issue will be fixed.
There is a more active discussion of a very similar issue in VFS-617. IMO
VFS-617 fixes fixes VFS-590 as well.
> SFTP moveTo operation might fail on permission checks even if the operation
> itself might succeed
>
>
> Key: VFS-590
> URL: https://issues.apache.org/jira/browse/VFS-590
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: Nightly Builds
>Reporter: L
>
> Continuing with SFTP moveTo, see VFS-588 and VFS-589 for more details.
> The permission checks that VFS performs during moveTo() can result in
> failure.
> org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(final FileObject
> destFile) starts with some checks:
> {code:java}
> @Override
> public void moveTo(final FileObject destFile) throws FileSystemException
> {
> if (canRenameTo(destFile))
> {
> if (!getParent().isWriteable())
> {
> throw new
> FileSystemException("vfs.provider/rename-parent-read-only.error",
> getName(),
> getParent().getName());
> }
> }
> else
> {
> if (!isWriteable())
> {
> throw new
> FileSystemException("vfs.provider/rename-read-only.error", getName());
> }
> }
> {code}
> The problem is: isWriteable() might fail or even hang. Performing moveTo
> without this check might successfully rename the file.
> isWriteable() in case of SFTP can fail if the server disables SSH
> channelExec. Or if it does not support "id" command.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (VFS-590) SFTP moveTo operation might fail on permission checks even if the operation itself might succeed
[
https://issues.apache.org/jira/browse/VFS-590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296810#comment-16296810
]
L commented on VFS-590:
---
I do not know how and when this issue will be fixed.
There is a more active discussion of a very similar issue in VFS-617. IMO
VFS-617 fixes fixes VFS-590 as well.
> SFTP moveTo operation might fail on permission checks even if the operation
> itself might succeed
>
>
> Key: VFS-590
> URL: https://issues.apache.org/jira/browse/VFS-590
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: Nightly Builds
>Reporter: L
>
> Continuing with SFTP moveTo, see VFS-588 and VFS-589 for more details.
> The permission checks that VFS performs during moveTo() can result in
> failure.
> org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(final FileObject
> destFile) starts with some checks:
> {code:java}
> @Override
> public void moveTo(final FileObject destFile) throws FileSystemException
> {
> if (canRenameTo(destFile))
> {
> if (!getParent().isWriteable())
> {
> throw new
> FileSystemException("vfs.provider/rename-parent-read-only.error",
> getName(),
> getParent().getName());
> }
> }
> else
> {
> if (!isWriteable())
> {
> throw new
> FileSystemException("vfs.provider/rename-read-only.error", getName());
> }
> }
> {code}
> The problem is: isWriteable() might fail or even hang. Performing moveTo
> without this check might successfully rename the file.
> isWriteable() in case of SFTP can fail if the server disables SSH
> channelExec. Or if it does not support "id" command.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (VFS-590) SFTP moveTo operation might fail on permission checks even if the operation itself might succeed
[
https://issues.apache.org/jira/browse/VFS-590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296786#comment-16296786
]
L commented on VFS-590:
---
As I wrote in the description:
{quote}Performing moveTo without this check might successfully rename the file.
{quote}
In our case it worked: the server had disabled SSH channelExec. After patching
commons-vfs not to perform all those isWriteable() moveTo() completed
successfully.
> SFTP moveTo operation might fail on permission checks even if the operation
> itself might succeed
>
>
> Key: VFS-590
> URL: https://issues.apache.org/jira/browse/VFS-590
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: Nightly Builds
>Reporter: L
>
> Continuing with SFTP moveTo, see VFS-588 and VFS-589 for more details.
> The permission checks that VFS performs during moveTo() can result in
> failure.
> org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(final FileObject
> destFile) starts with some checks:
> {code:java}
> @Override
> public void moveTo(final FileObject destFile) throws FileSystemException
> {
> if (canRenameTo(destFile))
> {
> if (!getParent().isWriteable())
> {
> throw new
> FileSystemException("vfs.provider/rename-parent-read-only.error",
> getName(),
> getParent().getName());
> }
> }
> else
> {
> if (!isWriteable())
> {
> throw new
> FileSystemException("vfs.provider/rename-read-only.error", getName());
> }
> }
> {code}
> The problem is: isWriteable() might fail or even hang. Performing moveTo
> without this check might successfully rename the file.
> isWriteable() in case of SFTP can fail if the server disables SSH
> channelExec. Or if it does not support "id" command.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (VFS-589) SFTP moveTo operation hangs if the server does not support SSH channelExec.
[
https://issues.apache.org/jira/browse/VFS-589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226410#comment-16226410
]
L commented on VFS-589:
---
The fix looks OK. As for verifying it: I have proposed a test to verify the
fix, see patch_sftp_tests_hang_no_exec.diff.
I think it is a good idea to add this test case to the set of SFTP tests. This
way you get the automatic verification.
> SFTP moveTo operation hangs if the server does not support SSH channelExec.
> ---
>
> Key: VFS-589
> URL: https://issues.apache.org/jira/browse/VFS-589
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: Nightly Builds
>Reporter: L
> Fix For: 2.3
>
> Attachments: patch_sftp_tests_hang_no_exec.diff,
> patch_sftp_timeouts.diff
>
>
> In our case the server was explicitly configured to disable SSH channelExec.
> Our code was hanging trying to execute moveTo(). Stacktrace:
> {code}
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.io.PipedInputStream.read(PipedInputStream.java:326)
> - locked <0x0006a7a184d0> (a
> com.jcraft.jsch.Channel$MyPipedInputStream)
> at java.io.PipedInputStream.read(PipedInputStream.java:377)
> - locked <0x0006a7a184d0> (a
> com.jcraft.jsch.Channel$MyPipedInputStream)
> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
> - locked <0x0006a7a184b8> (a java.io.InputStreamReader)
> at java.io.InputStreamReader.read(InputStreamReader.java:184)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileSystem.executeCommand(SftpFileSystem.java:328)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileSystem.getGroupsIds(SftpFileSystem.java:260)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileObject.getPermissions(SftpFileObject.java:317)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileObject.doIsWriteable(SftpFileObject.java:357)
> at
> org.apache.commons.vfs2.provider.AbstractFileObject.isWriteable(AbstractFileObject.java:1791)
> at
> org.apache.commons.vfs2.impl.DecoratedFileObject.isWriteable(DecoratedFileObject.java:229)
> at
> org.apache.commons.vfs2.cache.OnCallRefreshFileObject.isWriteable(OnCallRefreshFileObject.java:156)
> at
> org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1857)
> at
> org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241)
> at
> org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184)
> ...
> {code}
> Technically the connection was alive because the session had a configured
> timeout and the jcraft code kept sending keepalive SSH_MSG_GLOBAL_REQUEST
> messages, but the thread performing FileObject.moveTo() did not return from
> moveTo().
> I have changed SftpProviderTestCase to reproduce the problem:
> testRenameFile() hangs.
> The patch (patch_sftp_tests_hang_no_exec.diff) is attached.
> I traced the problem to the fact that VFS invokes method
> com.jcraft.jsch.Channel.connect(). This method uses timeout value 0, in which
> case class com.jcraft.jsch.ChannelExec creates an instance of class
> com.jcraft.jsch.RequestExec that sends an SSH packet SSH_MSG_CHANNEL_REQUEST
> with "want reply" set to 0.
> Correspondingly, if the server supports SSH channelExec, it executes the
> specified command and returns some data.
> But if the server *does not* support SSH channelExec it sends nothing back
> while jcraft code tries to read something. This is the hang I am observing.
> The fix would be to invoke com.jcraft.jsch.Channel.connect(int
> connectTimeout).
> As a result jcraft sends an SSH packet SSH_MSG_CHANNEL_REQUEST with "want
> reply" set to 1 *and* it waits for an answer *and* it reacts to the answer.
> Correspondingly, if the server supports SSH channelExec, it sends an SSH
> packet SSH_MSG_CHANNEL_SUCCESS and the executes the specified command and
> returns some data.
> If the server *does not* support SSH channelExec it sends an SSH packet
> SSH_MSG_CHANNEL_FAILURE.
> jcraft reacts on either of this messages because if waits for one of them. If
> it receives SSH_MSG_CHANNEL_SUCCESS it goes further and reads the response of
> the executed command.
> If it receives SSH_MSG_CHANNEL_FAILURE it immediately reports this by
> throwing JSchException with message "failed to send channel request".
> There is no hang whatsoever. Instead all tests from ProviderRenameTests fail
> with errors like
> {code}
> Could not determine if file "sftp://testtest@localhost:50036/write-tests; is
> writeable.
> {code}
> The test suite actually hangs at the
[jira] [Commented] (VFS-589) SFTP moveTo operation hangs if the server does not support SSH channelExec.
[
https://issues.apache.org/jira/browse/VFS-589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218961#comment-16218961
]
L commented on VFS-589:
---
Sorry, what "complexity"? diff file of 64 lines patching two java files?
Letting SftpFileSystemConfigBuilder have a default timeout would solve one
thing only: the proposed patch to SftpClientFactory.java.
It does not solve the *main* problem: SftpFileSystem.java invoking
channel.connect(); instead of channel.connect(some_timeout);
> SFTP moveTo operation hangs if the server does not support SSH channelExec.
> ---
>
> Key: VFS-589
> URL: https://issues.apache.org/jira/browse/VFS-589
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: Nightly Builds
>Reporter: L
> Attachments: patch_sftp_tests_hang_no_exec.diff,
> patch_sftp_timeouts.diff
>
>
> In our case the server was explicitly configured to disable SSH channelExec.
> Our code was hanging trying to execute moveTo(). Stacktrace:
> {code}
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.io.PipedInputStream.read(PipedInputStream.java:326)
> - locked <0x0006a7a184d0> (a
> com.jcraft.jsch.Channel$MyPipedInputStream)
> at java.io.PipedInputStream.read(PipedInputStream.java:377)
> - locked <0x0006a7a184d0> (a
> com.jcraft.jsch.Channel$MyPipedInputStream)
> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
> - locked <0x0006a7a184b8> (a java.io.InputStreamReader)
> at java.io.InputStreamReader.read(InputStreamReader.java:184)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileSystem.executeCommand(SftpFileSystem.java:328)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileSystem.getGroupsIds(SftpFileSystem.java:260)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileObject.getPermissions(SftpFileObject.java:317)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileObject.doIsWriteable(SftpFileObject.java:357)
> at
> org.apache.commons.vfs2.provider.AbstractFileObject.isWriteable(AbstractFileObject.java:1791)
> at
> org.apache.commons.vfs2.impl.DecoratedFileObject.isWriteable(DecoratedFileObject.java:229)
> at
> org.apache.commons.vfs2.cache.OnCallRefreshFileObject.isWriteable(OnCallRefreshFileObject.java:156)
> at
> org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1857)
> at
> org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241)
> at
> org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184)
> ...
> {code}
> Technically the connection was alive because the session had a configured
> timeout and the jcraft code kept sending keepalive SSH_MSG_GLOBAL_REQUEST
> messages, but the thread performing FileObject.moveTo() did not return from
> moveTo().
> I have changed SftpProviderTestCase to reproduce the problem:
> testRenameFile() hangs.
> The patch (patch_sftp_tests_hang_no_exec.diff) is attached.
> I traced the problem to the fact that VFS invokes method
> com.jcraft.jsch.Channel.connect(). This method uses timeout value 0, in which
> case class com.jcraft.jsch.ChannelExec creates an instance of class
> com.jcraft.jsch.RequestExec that sends an SSH packet SSH_MSG_CHANNEL_REQUEST
> with "want reply" set to 0.
> Correspondingly, if the server supports SSH channelExec, it executes the
> specified command and returns some data.
> But if the server *does not* support SSH channelExec it sends nothing back
> while jcraft code tries to read something. This is the hang I am observing.
> The fix would be to invoke com.jcraft.jsch.Channel.connect(int
> connectTimeout).
> As a result jcraft sends an SSH packet SSH_MSG_CHANNEL_REQUEST with "want
> reply" set to 1 *and* it waits for an answer *and* it reacts to the answer.
> Correspondingly, if the server supports SSH channelExec, it sends an SSH
> packet SSH_MSG_CHANNEL_SUCCESS and the executes the specified command and
> returns some data.
> If the server *does not* support SSH channelExec it sends an SSH packet
> SSH_MSG_CHANNEL_FAILURE.
> jcraft reacts on either of this messages because if waits for one of them. If
> it receives SSH_MSG_CHANNEL_SUCCESS it goes further and reads the response of
> the executed command.
> If it receives SSH_MSG_CHANNEL_FAILURE it immediately reports this by
> throwing JSchException with message "failed to send channel request".
> There is no hang whatsoever. Instead all tests from ProviderRenameTests fail
> with errors like
> {code}
> Could not determine if file
[jira] [Commented] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
[
https://issues.apache.org/jira/browse/VFS-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139746#comment-16139746
]
L commented on VFS-640:
---
Well, if it is called recursively for layered filesystems, it is *different*
provider, I guess.
And as for the original case: it was an SFTP provider where the server had a
lot of files (like more than 2000) in the root directory for the user.
> Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
> ---
>
> Key: VFS-640
> URL: https://issues.apache.org/jira/browse/VFS-640
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.0, 2.1
>Reporter: L
> Attachments: ParallelConnTest.java, patch_VFS640.diff
>
>
> Currently (rev. 1802455) this method looks like this:
> {code:java}
> protected synchronized FileSystem getFileSystem(final FileName rootName,
> final FileSystemOptions fileSystemOptions)
> throws FileSystemException
> {
> FileSystem fs = findFileSystem(rootName, fileSystemOptions);
> if (fs == null)
> {
> // Need to create the file system, and cache it
> fs = doCreateFileSystem(rootName, fileSystemOptions);
> addFileSystem(rootName, fs);
> }
> return fs;
> }
> {code}
> Given that there is a single instance of a provider per scheme, a very slow
> server that is being accessed first time will block *all* other threads
> trying to access some other resources via the same provider.
> We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We
> do not mind the thread trying to access this method taking that long, but
> because of that "synchronized" we end up having multiple threads accessing
> unrelated servers for the same scheme blocked.
> PS. Actually, it is not only AbstractOriginatingFileProvider. The same
> "provider-scoped" lock is present in AbstractLayeredFileProvider,
> TemporaryFileProvider, and UrlFileProvider
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
[
https://issues.apache.org/jira/browse/VFS-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139687#comment-16139687
]
L edited comment on VFS-640 at 8/24/17 7:19 AM:
Avoiding slow connection would be nice but unfortunately we do not have that
luxury.
I have created a patch for our internal use, see attachment. Some notes:
# The patch is based on 2.1 release and not on trunk.
# I tried to keep the code style but I did not verify it.
# It is indeed a lock keyed by the same key as the FileSystem. Its lifetime is
almost the same as lifetime of the corresponding FileSystem: it is created
before the FileSystem but is normally destroyed together with the FileSystem in
AbstractFileProvider.closeFileSystem()
# The lock is ref-counted to support scenarios when creating a FileSystem
fails. In this case the lock is destroyed when ref count goes to 0.
# All the lock management happens under in AbstractFileProvider synchronized
(fileSystems), no new synchronization objects are created.
# There is no deadlock. There are only two lock patterns: either "synchronized
(fileSystems)" or "synchronized (newLock), synchronized (fileSystems)"
# There is also a test (ParallelConnTest.java). The test has two methods, to
test the sequential behavior (the same key) and the parallel behavior
(different keys).
was (Author: klv_m72):
Avoiding slow connection would be nice but unfortunately we do not have that
luxury.
I have created a patch for our internal use, see attachment. Some notes:
# The patch is based on 2.1 release and not on trunk.
# I tried to keep the code style but I did not verify it.
# It is indeed a lock keyed by the same key as the FileSystem. Its lifetime is
almost the same as lifetime of the corresponding FileSystem: it is created
before the FileSystem but is normally destroyed together with the FileSystem in
AbstractFileProvider.closeFileSystem()
# The lock is ref-counted to support scenarios when creating a FileSystem
fails. In this case the lock is destroyed when ref count goes to 0.
# All the lock management happens under in AbstractFileProvider synchronized
(fileSystems), no new synchronization objects are created.
# There is no deadlock. There are only two lock patterns: either "synchronized
(fileSystems)" or "synchronized (newLock), synchronized (fileSystems)"
> Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
> ---
>
> Key: VFS-640
> URL: https://issues.apache.org/jira/browse/VFS-640
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.0, 2.1
>Reporter: L
> Attachments: ParallelConnTest.java, patch_VFS640.diff
>
>
> Currently (rev. 1802455) this method looks like this:
> {code:java}
> protected synchronized FileSystem getFileSystem(final FileName rootName,
> final FileSystemOptions fileSystemOptions)
> throws FileSystemException
> {
> FileSystem fs = findFileSystem(rootName, fileSystemOptions);
> if (fs == null)
> {
> // Need to create the file system, and cache it
> fs = doCreateFileSystem(rootName, fileSystemOptions);
> addFileSystem(rootName, fs);
> }
> return fs;
> }
> {code}
> Given that there is a single instance of a provider per scheme, a very slow
> server that is being accessed first time will block *all* other threads
> trying to access some other resources via the same provider.
> We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We
> do not mind the thread trying to access this method taking that long, but
> because of that "synchronized" we end up having multiple threads accessing
> unrelated servers for the same scheme blocked.
> PS. Actually, it is not only AbstractOriginatingFileProvider. The same
> "provider-scoped" lock is present in AbstractLayeredFileProvider,
> TemporaryFileProvider, and UrlFileProvider
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
[
https://issues.apache.org/jira/browse/VFS-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L updated VFS-640:
--
Attachment: patch_VFS640.diff
ParallelConnTest.java
patch_VFS640.diff: the patch
ParallelConnTest.java: the test
> Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
> ---
>
> Key: VFS-640
> URL: https://issues.apache.org/jira/browse/VFS-640
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.0, 2.1
>Reporter: L
> Attachments: ParallelConnTest.java, patch_VFS640.diff
>
>
> Currently (rev. 1802455) this method looks like this:
> {code:java}
> protected synchronized FileSystem getFileSystem(final FileName rootName,
> final FileSystemOptions fileSystemOptions)
> throws FileSystemException
> {
> FileSystem fs = findFileSystem(rootName, fileSystemOptions);
> if (fs == null)
> {
> // Need to create the file system, and cache it
> fs = doCreateFileSystem(rootName, fileSystemOptions);
> addFileSystem(rootName, fs);
> }
> return fs;
> }
> {code}
> Given that there is a single instance of a provider per scheme, a very slow
> server that is being accessed first time will block *all* other threads
> trying to access some other resources via the same provider.
> We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We
> do not mind the thread trying to access this method taking that long, but
> because of that "synchronized" we end up having multiple threads accessing
> unrelated servers for the same scheme blocked.
> PS. Actually, it is not only AbstractOriginatingFileProvider. The same
> "provider-scoped" lock is present in AbstractLayeredFileProvider,
> TemporaryFileProvider, and UrlFileProvider
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
[
https://issues.apache.org/jira/browse/VFS-640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139687#comment-16139687
]
L commented on VFS-640:
---
Avoiding slow connection would be nice but unfortunately we do not have that
luxury.
I have created a patch for our internal use, see attachment. Some notes:
# The patch is based on 2.1 release and not on trunk.
# I tried to keep the code style but I did not verify it.
# It is indeed a lock keyed by the same key as the FileSystem. Its lifetime is
almost the same as lifetime of the corresponding FileSystem: it is created
before the FileSystem but is normally destroyed together with the FileSystem in
AbstractFileProvider.closeFileSystem()
# The lock is ref-counted to support scenarios when creating a FileSystem
fails. In this case the lock is destroyed when ref count goes to 0.
# All the lock management happens under in AbstractFileProvider synchronized
(fileSystems), no new synchronization objects are created.
# There is no deadlock. There are only two lock patterns: either "synchronized
(fileSystems)" or "synchronized (newLock), synchronized (fileSystems)"
> Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
> ---
>
> Key: VFS-640
> URL: https://issues.apache.org/jira/browse/VFS-640
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.0, 2.1
>Reporter: L
>
> Currently (rev. 1802455) this method looks like this:
> {code:java}
> protected synchronized FileSystem getFileSystem(final FileName rootName,
> final FileSystemOptions fileSystemOptions)
> throws FileSystemException
> {
> FileSystem fs = findFileSystem(rootName, fileSystemOptions);
> if (fs == null)
> {
> // Need to create the file system, and cache it
> fs = doCreateFileSystem(rootName, fileSystemOptions);
> addFileSystem(rootName, fs);
> }
> return fs;
> }
> {code}
> Given that there is a single instance of a provider per scheme, a very slow
> server that is being accessed first time will block *all* other threads
> trying to access some other resources via the same provider.
> We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We
> do not mind the thread trying to access this method taking that long, but
> because of that "synchronized" we end up having multiple threads accessing
> unrelated servers for the same scheme blocked.
> PS. Actually, it is not only AbstractOriginatingFileProvider. The same
> "provider-scoped" lock is present in AbstractLayeredFileProvider,
> TemporaryFileProvider, and UrlFileProvider
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
[
https://issues.apache.org/jira/browse/VFS-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L updated VFS-640:
--
Description:
Currently (rev. 1802455) this method looks like this:
{code:java}
protected synchronized FileSystem getFileSystem(final FileName rootName,
final FileSystemOptions fileSystemOptions)
throws FileSystemException
{
FileSystem fs = findFileSystem(rootName, fileSystemOptions);
if (fs == null)
{
// Need to create the file system, and cache it
fs = doCreateFileSystem(rootName, fileSystemOptions);
addFileSystem(rootName, fs);
}
return fs;
}
{code}
Given that there is a single instance of a provider per scheme, a very slow
server that is being accessed first time will block *all* other threads trying
to access some other resources via the same provider.
We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We
do not mind the thread trying to access this method taking that long, but
because of that "synchronized" we end up having multiple threads accessing
unrelated servers for the same scheme blocked.
PS. Actually, it is not only AbstractOriginatingFileProvider. The same
"provider-scoped" lock is present in AbstractLayeredFileProvider,
TemporaryFileProvider, and UrlFileProvider
was:
Currently (rev. 1802455) this method looks like this:
{code:java}
protected synchronized FileSystem getFileSystem(final FileName rootName,
final FileSystemOptions fileSystemOptions)
throws FileSystemException
{
FileSystem fs = findFileSystem(rootName, fileSystemOptions);
if (fs == null)
{
// Need to create the file system, and cache it
fs = doCreateFileSystem(rootName, fileSystemOptions);
addFileSystem(rootName, fs);
}
return fs;
}
{code}
Given that there is a single instance of a provider per scheme, a very slow
server that is being accessed first time will block *all* other threads trying
to access some other resources via the same provider.
We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We
do not mind the thread trying to access this method taking that long, but
because of that "synchronized" we end up having multiple threads accessing
unrelated servers for the same scheme blocked.
Actually, it is not only
> Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
> ---
>
> Key: VFS-640
> URL: https://issues.apache.org/jira/browse/VFS-640
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.0, 2.1
>Reporter: L
>
> Currently (rev. 1802455) this method looks like this:
> {code:java}
> protected synchronized FileSystem getFileSystem(final FileName rootName,
> final FileSystemOptions fileSystemOptions)
> throws FileSystemException
> {
> FileSystem fs = findFileSystem(rootName, fileSystemOptions);
> if (fs == null)
> {
> // Need to create the file system, and cache it
> fs = doCreateFileSystem(rootName, fileSystemOptions);
> addFileSystem(rootName, fs);
> }
> return fs;
> }
> {code}
> Given that there is a single instance of a provider per scheme, a very slow
> server that is being accessed first time will block *all* other threads
> trying to access some other resources via the same provider.
> We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We
> do not mind the thread trying to access this method taking that long, but
> because of that "synchronized" we end up having multiple threads accessing
> unrelated servers for the same scheme blocked.
> PS. Actually, it is not only AbstractOriginatingFileProvider. The same
> "provider-scoped" lock is present in AbstractLayeredFileProvider,
> TemporaryFileProvider, and UrlFileProvider
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
[
https://issues.apache.org/jira/browse/VFS-640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L updated VFS-640:
--
Description:
Currently (rev. 1802455) this method looks like this:
{code:java}
protected synchronized FileSystem getFileSystem(final FileName rootName,
final FileSystemOptions fileSystemOptions)
throws FileSystemException
{
FileSystem fs = findFileSystem(rootName, fileSystemOptions);
if (fs == null)
{
// Need to create the file system, and cache it
fs = doCreateFileSystem(rootName, fileSystemOptions);
addFileSystem(rootName, fs);
}
return fs;
}
{code}
Given that there is a single instance of a provider per scheme, a very slow
server that is being accessed first time will block *all* other threads trying
to access some other resources via the same provider.
We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We
do not mind the thread trying to access this method taking that long, but
because of that "synchronized" we end up having multiple threads accessing
unrelated servers for the same scheme blocked.
Actually, it is not only
was:
Currently (rev. 1802455) this method looks like this:
{code:java}
protected synchronized FileSystem getFileSystem(final FileName rootName,
final FileSystemOptions fileSystemOptions)
throws FileSystemException
{
FileSystem fs = findFileSystem(rootName, fileSystemOptions);
if (fs == null)
{
// Need to create the file system, and cache it
fs = doCreateFileSystem(rootName, fileSystemOptions);
addFileSystem(rootName, fs);
}
return fs;
}
{code}
Given that there is a single instance of a provider per scheme, a very slow
server that is being accessed first time will block *all* other threads trying
to access some other resources via the same provider.
We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We
do not mind the thread trying to access this method taking that long, but
because of that "synchronized" we end up having multiple threads accessing
unrelated servers for the same scheme blocked.
> Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
> ---
>
> Key: VFS-640
> URL: https://issues.apache.org/jira/browse/VFS-640
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.0, 2.1
>Reporter: L
>
> Currently (rev. 1802455) this method looks like this:
> {code:java}
> protected synchronized FileSystem getFileSystem(final FileName rootName,
> final FileSystemOptions fileSystemOptions)
> throws FileSystemException
> {
> FileSystem fs = findFileSystem(rootName, fileSystemOptions);
> if (fs == null)
> {
> // Need to create the file system, and cache it
> fs = doCreateFileSystem(rootName, fileSystemOptions);
> addFileSystem(rootName, fs);
> }
> return fs;
> }
> {code}
> Given that there is a single instance of a provider per scheme, a very slow
> server that is being accessed first time will block *all* other threads
> trying to access some other resources via the same provider.
> We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We
> do not mind the thread trying to access this method taking that long, but
> because of that "synchronized" we end up having multiple threads accessing
> unrelated servers for the same scheme blocked.
> Actually, it is not only
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (VFS-640) Synchronized AbstractOriginatingFileProvider.getFileSystem is too broad
L created VFS-640:
-
Summary: Synchronized
AbstractOriginatingFileProvider.getFileSystem is too broad
Key: VFS-640
URL: https://issues.apache.org/jira/browse/VFS-640
Project: Commons VFS
Issue Type: Bug
Affects Versions: 2.1, 2.0
Reporter: L
Currently (rev. 1802455) this method looks like this:
{code:java}
protected synchronized FileSystem getFileSystem(final FileName rootName,
final FileSystemOptions fileSystemOptions)
throws FileSystemException
{
FileSystem fs = findFileSystem(rootName, fileSystemOptions);
if (fs == null)
{
// Need to create the file system, and cache it
fs = doCreateFileSystem(rootName, fileSystemOptions);
addFileSystem(rootName, fs);
}
return fs;
}
{code}
Given that there is a single instance of a provider per scheme, a very slow
server that is being accessed first time will block *all* other threads trying
to access some other resources via the same provider.
We have a server that takes sometimes 20-30 seconds in doCreateFileSystem. We
do not mind the thread trying to access this method taking that long, but
because of that "synchronized" we end up having multiple threads accessing
unrelated servers for the same scheme blocked.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (VFS-617) isReadable fails if unable to determine group identity
[ https://issues.apache.org/jira/browse/VFS-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956494#comment-15956494 ] L commented on VFS-617: --- The thing is: this operation would always fail, no matter where it is invoked, on move() or on login, if exec channel is not allowed or id command is not present. I actually was forced to patch commons-vfs locally. I have added a new SFTP configuration option in class SftpFileSystemConfigBuilder, which I called hardcodedRWXPermissions. When true, doIsExecutable/doIsReadable/doIsWriteable always return true. This way I avoid executing "id". Instead I rely on jssh to report errors if the actual file operation fails. Works like a charm. > isReadable fails if unable to determine group identity > -- > > Key: VFS-617 > URL: https://issues.apache.org/jira/browse/VFS-617 > Project: Commons VFS > Issue Type: Bug >Affects Versions: 2.1 > Environment: Windows 7 Java 7. Failure occured connecting via SFTP to > a Synology box running DSM 6. >Reporter: Tim Nickels > > The doIsReadable method of SftpFileObject throws an exception if the system > cannot identify group/owner permissions... > Exception in thread "main" org.apache.commons.vfs2.FileSystemException: Could > not determine if file "sftp://myURI; is readable. > at > org.apache.commons.vfs2.provider.AbstractFileObject.isReadable(AbstractFileObject.java:1761) > at com.avenca.vfs.VFSUtils.main(VFSUtils.java:41) > Caused by: com.jcraft.jsch.JSchException: Could not get the groups id of the > current user (error code: 1) > at > org.apache.commons.vfs2.provider.sftp.SftpFileSystem.getGroupsIds(SftpFileSystem.java:263) > at > org.apache.commons.vfs2.provider.sftp.SftpFileObject.getPermissions(SftpFileObject.java:317) > at > org.apache.commons.vfs2.provider.sftp.SftpFileObject.doIsReadable(SftpFileObject.java:335) > at > org.apache.commons.vfs2.provider.AbstractFileObject.isReadable(AbstractFileObject.java:1757) > The problem is the method is using > return getPermissions(true).isReadable() > The folder *is* readable without these permissions, and so should be set to > return getPermissions(false).isReadable() > Which correctly allows the system to identify a readable folder without > adding unnecessary restrictions. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (VFS-617) isReadable fails if unable to determine group identity
[ https://issues.apache.org/jira/browse/VFS-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956457#comment-15956457 ] L commented on VFS-617: --- Looks like a duplicate of VFS-590. > isReadable fails if unable to determine group identity > -- > > Key: VFS-617 > URL: https://issues.apache.org/jira/browse/VFS-617 > Project: Commons VFS > Issue Type: Bug >Affects Versions: 2.1 > Environment: Windows 7 Java 7. Failure occured connecting via SFTP to > a Synology box running DSM 6. >Reporter: Tim Nickels > > The doIsReadable method of SftpFileObject throws an exception if the system > cannot identify group/owner permissions... > Exception in thread "main" org.apache.commons.vfs2.FileSystemException: Could > not determine if file "sftp://myURI; is readable. > at > org.apache.commons.vfs2.provider.AbstractFileObject.isReadable(AbstractFileObject.java:1761) > at com.avenca.vfs.VFSUtils.main(VFSUtils.java:41) > Caused by: com.jcraft.jsch.JSchException: Could not get the groups id of the > current user (error code: 1) > at > org.apache.commons.vfs2.provider.sftp.SftpFileSystem.getGroupsIds(SftpFileSystem.java:263) > at > org.apache.commons.vfs2.provider.sftp.SftpFileObject.getPermissions(SftpFileObject.java:317) > at > org.apache.commons.vfs2.provider.sftp.SftpFileObject.doIsReadable(SftpFileObject.java:335) > at > org.apache.commons.vfs2.provider.AbstractFileObject.isReadable(AbstractFileObject.java:1757) > The problem is the method is using > return getPermissions(true).isReadable() > The folder *is* readable without these permissions, and so should be set to > return getPermissions(false).isReadable() > Which correctly allows the system to identify a readable folder without > adding unnecessary restrictions. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (VFS-630) Sftp code assumes that exec channels are available and *nix server
[ https://issues.apache.org/jira/browse/VFS-630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15956456#comment-15956456 ] L commented on VFS-630: --- It is a duplicate of VFS-590. > Sftp code assumes that exec channels are available and *nix server > -- > > Key: VFS-630 > URL: https://issues.apache.org/jira/browse/VFS-630 > Project: Commons VFS > Issue Type: Bug >Affects Versions: 2.1 > Environment: Any client running VFS 2.1/JSch 1.51 and later using > remote sftp server that's locked down or not *nix based. >Reporter: Will Wood >Priority: Minor > > In working with VFS using SFTP there's an assumption in the code that creates > a problem if the remote server is locked down or it's not *nix based. I hit > this when working with a server that had exec commands disabled and > attempting to do a moveTo command on a remote file object renaming it to the > same server as a remote file object. At one point the VFS code attempts to > execute an "id -G" command on a JSch "exec" channel. Since the remote server > disallows the exec command the subsequent read from the input stream blocks > until the O/S default socket timeout occurs, it basically hangs. > I tested this same scenario against a Linux server, it worked fine. I also > tested a Windows server which doesn't have an id command, it failed. > Since JSch deals with this natively, I would recommend getting rid of the > exec commands altogether and letting the client deal with those issues > outside of the framework, i.e., moveTo in this case doesn't need to worry > about permissions but rather throws the underlying exceptions from JSch. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (VFS-589) SFTP moveTo operation hangs if the server does not support SSH channelExec.
L created VFS-589:
-
Summary: SFTP moveTo operation hangs if the server does not
support SSH channelExec.
Key: VFS-589
URL: https://issues.apache.org/jira/browse/VFS-589
Project: Commons VFS
Issue Type: Bug
Affects Versions: Nightly Builds
Reporter: L
In our case the server was explicitly configured to disable SSH channelExec.
Our code was hanging trying to execute moveTo(). Stacktrace:
{code}
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.io.PipedInputStream.read(PipedInputStream.java:326)
- locked <0x0006a7a184d0> (a com.jcraft.jsch.Channel$MyPipedInputStream)
at java.io.PipedInputStream.read(PipedInputStream.java:377)
- locked <0x0006a7a184d0> (a com.jcraft.jsch.Channel$MyPipedInputStream)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
- locked <0x0006a7a184b8> (a java.io.InputStreamReader)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at
org.apache.commons.vfs2.provider.sftp.SftpFileSystem.executeCommand(SftpFileSystem.java:328)
at
org.apache.commons.vfs2.provider.sftp.SftpFileSystem.getGroupsIds(SftpFileSystem.java:260)
at
org.apache.commons.vfs2.provider.sftp.SftpFileObject.getPermissions(SftpFileObject.java:317)
at
org.apache.commons.vfs2.provider.sftp.SftpFileObject.doIsWriteable(SftpFileObject.java:357)
at
org.apache.commons.vfs2.provider.AbstractFileObject.isWriteable(AbstractFileObject.java:1791)
at
org.apache.commons.vfs2.impl.DecoratedFileObject.isWriteable(DecoratedFileObject.java:229)
at
org.apache.commons.vfs2.cache.OnCallRefreshFileObject.isWriteable(OnCallRefreshFileObject.java:156)
at
org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1857)
at
org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241)
at
org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184)
...
{code}
Technically the connection was alive because the session had a configured
timeout and the jcraft code kept sending keepalive SSH_MSG_GLOBAL_REQUEST
messages, but the thread performing FileObject.moveTo() did not return from
moveTo().
I have changed SftpProviderTestCase to reproduce the problem: testRenameFile()
hangs.
The patch (patch_sftp_tests_hang_no_exec.diff) is attached.
I traced the problem to the fact that VFS invokes method
com.jcraft.jsch.Channel.connect(). This method uses timeout value 0, in which
case class com.jcraft.jsch.ChannelExec creates an instance of class
com.jcraft.jsch.RequestExec that sends an SSH packet SSH_MSG_CHANNEL_REQUEST
with "want reply" set to 0.
Correspondingly, if the server supports SSH channelExec, it executes the
specified command and returns some data.
But if the server *does not* support SSH channelExec it sends nothing back
while jcraft code tries to read something. This is the hang I am observing.
The fix would be to invoke com.jcraft.jsch.Channel.connect(int connectTimeout).
As a result jcraft sends an SSH packet SSH_MSG_CHANNEL_REQUEST with "want
reply" set to 1 *and* it waits for an answer *and* it reacts to the answer.
Correspondingly, if the server supports SSH channelExec, it sends an SSH packet
SSH_MSG_CHANNEL_SUCCESS and the executes the specified command and returns some
data.
If the server *does not* support SSH channelExec it sends an SSH packet
SSH_MSG_CHANNEL_FAILURE.
jcraft reacts on either of this messages because if waits for one of them. If
it receives SSH_MSG_CHANNEL_SUCCESS it goes further and reads the response of
the executed command.
If it receives SSH_MSG_CHANNEL_FAILURE it immediately reports this by throwing
JSchException with message "failed to send channel request".
There is no hang whatsoever. Instead all tests from ProviderRenameTests fail
with errors like
{code}
Could not determine if file "sftp://testtest@localhost:50036/write-tests; is
writeable.
{code}
The test suite actually hangs at the end, but this is caused by
https://issues.apache.org/jira/browse/VFS-588
I have patched VFS classes to always open jcraft's channels with timeouts. In
addition the patch always sets some default timeout value on jcraft's session
if none was configured via SftpFileSystemConfigBuilder.
Patch is also attached: patch_sftp_timeouts.diff
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (VFS-590) SFTP moveTo operation might fail on permission checks even if the operation itself might succeed
L created VFS-590:
-
Summary: SFTP moveTo operation might fail on permission checks
even if the operation itself might succeed
Key: VFS-590
URL: https://issues.apache.org/jira/browse/VFS-590
Project: Commons VFS
Issue Type: Bug
Affects Versions: Nightly Builds
Reporter: L
Continuing with SFTP moveTo, see VFS-588 and VFS-589 for more details.
The permission checks that VFS performs during moveTo() can result in failure.
org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(final FileObject
destFile) starts with some checks:
{code:java}
@Override
public void moveTo(final FileObject destFile) throws FileSystemException
{
if (canRenameTo(destFile))
{
if (!getParent().isWriteable())
{
throw new
FileSystemException("vfs.provider/rename-parent-read-only.error",
getName(),
getParent().getName());
}
}
else
{
if (!isWriteable())
{
throw new
FileSystemException("vfs.provider/rename-read-only.error", getName());
}
}
{code}
The problem is: isWriteable() might fail or even hang while performing moveTo
without this check might successfully rename the file.
isWriteable() in case of SFTP can fail if the server disables SSH channelExec.
Or if it does not support "id" command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (VFS-590) SFTP moveTo operation might fail on permission checks even if the operation itself might succeed
[
https://issues.apache.org/jira/browse/VFS-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L updated VFS-590:
--
Description:
Continuing with SFTP moveTo, see VFS-588 and VFS-589 for more details.
The permission checks that VFS performs during moveTo() can result in failure.
org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(final FileObject
destFile) starts with some checks:
{code:java}
@Override
public void moveTo(final FileObject destFile) throws FileSystemException
{
if (canRenameTo(destFile))
{
if (!getParent().isWriteable())
{
throw new
FileSystemException("vfs.provider/rename-parent-read-only.error",
getName(),
getParent().getName());
}
}
else
{
if (!isWriteable())
{
throw new
FileSystemException("vfs.provider/rename-read-only.error", getName());
}
}
{code}
The problem is: isWriteable() might fail or even hang. Performing moveTo
without this check might successfully rename the file.
isWriteable() in case of SFTP can fail if the server disables SSH channelExec.
Or if it does not support "id" command.
was:
Continuing with SFTP moveTo, see VFS-588 and VFS-589 for more details.
The permission checks that VFS performs during moveTo() can result in failure.
org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(final FileObject
destFile) starts with some checks:
{code:java}
@Override
public void moveTo(final FileObject destFile) throws FileSystemException
{
if (canRenameTo(destFile))
{
if (!getParent().isWriteable())
{
throw new
FileSystemException("vfs.provider/rename-parent-read-only.error",
getName(),
getParent().getName());
}
}
else
{
if (!isWriteable())
{
throw new
FileSystemException("vfs.provider/rename-read-only.error", getName());
}
}
{code}
The problem is: isWriteable() might fail or even hang while performing moveTo
without this check might successfully rename the file.
isWriteable() in case of SFTP can fail if the server disables SSH channelExec.
Or if it does not support "id" command.
> SFTP moveTo operation might fail on permission checks even if the operation
> itself might succeed
>
>
> Key: VFS-590
> URL: https://issues.apache.org/jira/browse/VFS-590
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: Nightly Builds
>Reporter: L
>
> Continuing with SFTP moveTo, see VFS-588 and VFS-589 for more details.
> The permission checks that VFS performs during moveTo() can result in
> failure.
> org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(final FileObject
> destFile) starts with some checks:
> {code:java}
> @Override
> public void moveTo(final FileObject destFile) throws FileSystemException
> {
> if (canRenameTo(destFile))
> {
> if (!getParent().isWriteable())
> {
> throw new
> FileSystemException("vfs.provider/rename-parent-read-only.error",
> getName(),
> getParent().getName());
> }
> }
> else
> {
> if (!isWriteable())
> {
> throw new
> FileSystemException("vfs.provider/rename-read-only.error", getName());
> }
> }
> {code}
> The problem is: isWriteable() might fail or even hang. Performing moveTo
> without this check might successfully rename the file.
> isWriteable() in case of SFTP can fail if the server disables SSH
> channelExec. Or if it does not support "id" command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (VFS-589) SFTP moveTo operation hangs if the server does not support SSH channelExec.
[
https://issues.apache.org/jira/browse/VFS-589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L updated VFS-589:
--
Attachment: patch_sftp_tests_hang_no_exec.diff
Patch to the test (SftpProviderTestCase) to reproduce the issue
> SFTP moveTo operation hangs if the server does not support SSH channelExec.
> ---
>
> Key: VFS-589
> URL: https://issues.apache.org/jira/browse/VFS-589
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: Nightly Builds
>Reporter: L
> Attachments: patch_sftp_tests_hang_no_exec.diff,
> patch_sftp_timeouts.diff
>
>
> In our case the server was explicitly configured to disable SSH channelExec.
> Our code was hanging trying to execute moveTo(). Stacktrace:
> {code}
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.io.PipedInputStream.read(PipedInputStream.java:326)
> - locked <0x0006a7a184d0> (a
> com.jcraft.jsch.Channel$MyPipedInputStream)
> at java.io.PipedInputStream.read(PipedInputStream.java:377)
> - locked <0x0006a7a184d0> (a
> com.jcraft.jsch.Channel$MyPipedInputStream)
> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
> - locked <0x0006a7a184b8> (a java.io.InputStreamReader)
> at java.io.InputStreamReader.read(InputStreamReader.java:184)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileSystem.executeCommand(SftpFileSystem.java:328)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileSystem.getGroupsIds(SftpFileSystem.java:260)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileObject.getPermissions(SftpFileObject.java:317)
> at
> org.apache.commons.vfs2.provider.sftp.SftpFileObject.doIsWriteable(SftpFileObject.java:357)
> at
> org.apache.commons.vfs2.provider.AbstractFileObject.isWriteable(AbstractFileObject.java:1791)
> at
> org.apache.commons.vfs2.impl.DecoratedFileObject.isWriteable(DecoratedFileObject.java:229)
> at
> org.apache.commons.vfs2.cache.OnCallRefreshFileObject.isWriteable(OnCallRefreshFileObject.java:156)
> at
> org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1857)
> at
> org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241)
> at
> org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184)
> ...
> {code}
> Technically the connection was alive because the session had a configured
> timeout and the jcraft code kept sending keepalive SSH_MSG_GLOBAL_REQUEST
> messages, but the thread performing FileObject.moveTo() did not return from
> moveTo().
> I have changed SftpProviderTestCase to reproduce the problem:
> testRenameFile() hangs.
> The patch (patch_sftp_tests_hang_no_exec.diff) is attached.
> I traced the problem to the fact that VFS invokes method
> com.jcraft.jsch.Channel.connect(). This method uses timeout value 0, in which
> case class com.jcraft.jsch.ChannelExec creates an instance of class
> com.jcraft.jsch.RequestExec that sends an SSH packet SSH_MSG_CHANNEL_REQUEST
> with "want reply" set to 0.
> Correspondingly, if the server supports SSH channelExec, it executes the
> specified command and returns some data.
> But if the server *does not* support SSH channelExec it sends nothing back
> while jcraft code tries to read something. This is the hang I am observing.
> The fix would be to invoke com.jcraft.jsch.Channel.connect(int
> connectTimeout).
> As a result jcraft sends an SSH packet SSH_MSG_CHANNEL_REQUEST with "want
> reply" set to 1 *and* it waits for an answer *and* it reacts to the answer.
> Correspondingly, if the server supports SSH channelExec, it sends an SSH
> packet SSH_MSG_CHANNEL_SUCCESS and the executes the specified command and
> returns some data.
> If the server *does not* support SSH channelExec it sends an SSH packet
> SSH_MSG_CHANNEL_FAILURE.
> jcraft reacts on either of this messages because if waits for one of them. If
> it receives SSH_MSG_CHANNEL_SUCCESS it goes further and reads the response of
> the executed command.
> If it receives SSH_MSG_CHANNEL_FAILURE it immediately reports this by
> throwing JSchException with message "failed to send channel request".
> There is no hang whatsoever. Instead all tests from ProviderRenameTests fail
> with errors like
> {code}
> Could not determine if file "sftp://testtest@localhost:50036/write-tests; is
> writeable.
> {code}
> The test suite actually hangs at the end, but this is caused by
> https://issues.apache.org/jira/browse/VFS-588
> I have patched VFS classes to always open jcraft's channels with timeouts. In
> addition the patch always sets some
[jira] [Created] (VFS-588) SftpProviderTestCase is not robbust in case a timeout is configured
L created VFS-588: - Summary: SftpProviderTestCase is not robbust in case a timeout is configured Key: VFS-588 URL: https://issues.apache.org/jira/browse/VFS-588 Project: Commons VFS Issue Type: Bug Affects Versions: Nightly Builds Reporter: L Priority: Minor When a timeout is configured via SftpFileSystemConfigBuilder.setTimeout(), the test execution of SftpProviderTestCase hangs and does not stop. The hang is cause by "stream proxy test suite". Running SftpProviderTestCase with disabled "stream proxy test suite" does not hang and JVM stops. This might look really minor but I am about to file another issue (which is not that minor) where timeouts play an important role. Not being able to execute tests with timeouts is quite a showstopper for debugging that other issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (VFS-588) SftpProviderTestCase is not robbust in case a timeout is configured
[ https://issues.apache.org/jira/browse/VFS-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L updated VFS-588: -- Attachment: patch_hang_with_tout.diff The patch that enables timeout configuration. > SftpProviderTestCase is not robbust in case a timeout is configured > --- > > Key: VFS-588 > URL: https://issues.apache.org/jira/browse/VFS-588 > Project: Commons VFS > Issue Type: Bug >Affects Versions: Nightly Builds >Reporter: L >Priority: Minor > Attachments: patch_hang_with_tout.diff > > > When a timeout is configured via SftpFileSystemConfigBuilder.setTimeout(), > the test execution of SftpProviderTestCase hangs and does not stop. > The hang is cause by "stream proxy test suite". Running SftpProviderTestCase > with disabled "stream proxy test suite" does not hang and JVM stops. > This might look really minor but I am about to file another issue (which is > not that minor) where timeouts play an important role. Not being able to > execute tests with timeouts is quite a showstopper for debugging that other > issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (VFS-559) FTPClientWrapper is not robust against some failures
[
https://issues.apache.org/jira/browse/VFS-559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15029007#comment-15029007
]
L commented on VFS-559:
---
Yes, the error is fixed.
> FTPClientWrapper is not robust against some failures
>
>
> Key: VFS-559
> URL: https://issues.apache.org/jira/browse/VFS-559
> Project: Commons VFS
> Issue Type: Bug
>Affects Versions: 2.0
>Reporter: L
>
> The goal of the class is stated in javadoc:
> A wrapper to the FTPClient to allow automatic reconnect on connection loss.
> A lot of its methods look like :
> try
> {
> do something...
> }
> catch (final IOException e)
> {
> disconnect();
> try to repeat the operation...
> }
> Unfortunately disonnect() can fail for the same reason as the original "do
> something". In my case it as a "connection reset". So instead of the original
> exception I was getting more or less the same exception from
> getFtpClient().quit();
> So the wrapper did not help at all.
> I guess all the disconnect() invocations must also be inside try/catch so
> that even if disconnect() throws, the method goes on to the next step: try
> to repeat the operation...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (VFS-580) Improve FTP error messages
L created VFS-580:
-
Summary: Improve FTP error messages
Key: VFS-580
URL: https://issues.apache.org/jira/browse/VFS-580
Project: Commons VFS
Issue Type: Improvement
Affects Versions: Nightly Builds, 2.1
Reporter: L
FTP provider classes throw FileSystemException is some operations cannot be
performed. It would be really nice if the exceptions also included
ftpClient.getReplyString() in the message.
Reason: normally an error is reported to a user by using getMessage() on the
exception instance. The errors reported by FTP provider look now too generic
and cannot help diagnose the issue.
For example, FtpFileObject.doCreateFolder() has this code:
if (!ok)
{
throw new
FileSystemException("vfs.provider.ftp/create-folder.error", getName());
}
The reported error looks like:
Could not create FTP directory dir_name.
Why the operation failed? Only the stack trace can help.
There are of course more places where adding ftpClient.getReplyString() makes
sense.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (VFS-581) FTPClientWrapper hides important error information
L created VFS-581: - Summary: FTPClientWrapper hides important error information Key: VFS-581 URL: https://issues.apache.org/jira/browse/VFS-581 Project: Commons VFS Issue Type: Bug Affects Versions: Nightly Builds, 2.1 Reporter: L One place where FTPClientWrapper hides errors is method listFilesInDirectory(). 1. Lines 163-167: After performing getFtpClient().listFiles(relPath); the code checks for a positive FTP reply. 2. If the reply was negative, the code assumes the negative reply has to do with VFS-307 as stated in the comments. This is not always the case, there might be other reasons why the operation might have failed. In my case it was an FTPS connection where the server required an encrypted data channel and the client did was not properly configured. 3. Line 182: getFtpClient().listFiles();. This time without even looking at the FTP reply string. In my case it was the same error as above. As a result my client got an empty as if the remote directory did not have any files, while the real error reason was nowhere to see. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (VFS-558) java.lang.UnsupportedOperationException in FtpFileObject
[ https://issues.apache.org/jira/browse/VFS-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14283917#comment-14283917 ] L commented on VFS-558: --- Re: Thanks for testing! I really hope to release soon (so you better not find new bugs ) Sorry: VFS-559 java.lang.UnsupportedOperationException in FtpFileObject Key: VFS-558 URL: https://issues.apache.org/jira/browse/VFS-558 Project: Commons VFS Issue Type: Bug Affects Versions: 2.0 Reporter: L Assignee: Bernd Eckenfels Fix For: 2.1 I am getting the following exception in my code: java.lang.UnsupportedOperationException at java.util.Collections$UnmodifiableMap.remove(Collections.java:1345) at org.apache.commons.vfs2.provider.ftp.FtpFileObject.onChildrenChanged(FtpFileObject.java:271) at org.apache.commons.vfs2.provider.AbstractFileObject.childrenChanged(AbstractFileObject.java:240) at org.apache.commons.vfs2.provider.AbstractFileObject.notifyParent(AbstractFileObject.java:1931) at org.apache.commons.vfs2.provider.AbstractFileObject.handleCreate(AbstractFileObject.java:1577) at org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1866) at org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241) at org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184) ... I guess it is caused by the fact that children field is set to EMPTY_FTP_FILE_MAP at the moment onChildrenChanged() is invoked. I also do not like line 1866 in AbstractFileObject.java. To me it looks like it might be the real cause of the problem: FileObjectUtils.getAbstractFileObject(destFile).handleCreate(getType()); Must it not be destFile.getType()? But even if I am right about AbstractFileObject.java:1866, FtpFileObject.onChildrenChanged() must be corrected as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (VFS-559) FTPClientWrapper is not robust against some failures
L created VFS-559:
-
Summary: FTPClientWrapper is not robust against some failures
Key: VFS-559
URL: https://issues.apache.org/jira/browse/VFS-559
Project: Commons VFS
Issue Type: Bug
Affects Versions: 2.0
Reporter: L
The goal of the class is stated in javadoc:
A wrapper to the FTPClient to allow automatic reconnect on connection loss.
A lot of its methods look like :
try
{
do something...
}
catch (final IOException e)
{
disconnect();
try to repeat the operation...
}
Unfortunately disonnect() can fail for the same reason as the original do
something. In my case it as a connection reset. So instead of the original
exception I was getting more or less the same exception from
getFtpClient().quit();
So the wrapper did not help at all.
I guess all the disconnect() invocations must also be inside try/catch so that
even if disconnect() throws, the method goes on to the next step: try to
repeat the operation...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (VFS-558) java.lang.UnsupportedOperationException in FtpFileObject
[ https://issues.apache.org/jira/browse/VFS-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282815#comment-14282815 ] L commented on VFS-558: --- Re: Anyway, I fixed the rename in trunk, can you try? Yes, it is fixed, thanks. Re: I would use ON_RESOLVE or even MANUAL and then use refresh() in those particular places where you want to have an up-to-date view of the children. Because with ON_CALL you cannot control which of the methods actually need the refresh and which not. But: this should not be related to your bug report, I just noticed it. Since I was getting problems even with ON_CALL, I have added refresh() before checking if a file exists, and it looks like it has solved spurious warnings. I cannot be 100% sure because the warnings did not always occur when I have run my tests without refresh(). Re: BTW2: do you use OnCallRefresher on purpose, I can imagine that makes FTP quite slow. Re: and really consider not using ON_CALL This sounds intuitively as a correct piece of advice. The thing is: my tests show different results: I have a code that initializes an instance of StandardFileSystemManager with a setCacheStrategy(CacheStrategy.ON_CALL) before .init(). My tests accessing a local FS with setCacheStrategy(CacheStrategy.ON_CALL) commented out are about 40% faster than with setCacheStrategy(CacheStrategy.ON_CALL) performed. As expected. But FTP tests with ON_CALL are about the same 40% faster than with setCacheStrategy(CacheStrategy.ON_CALL) commented out. Dare to explain? java.lang.UnsupportedOperationException in FtpFileObject Key: VFS-558 URL: https://issues.apache.org/jira/browse/VFS-558 Project: Commons VFS Issue Type: Bug Affects Versions: 2.0 Reporter: L Assignee: Bernd Eckenfels Fix For: 2.1 I am getting the following exception in my code: java.lang.UnsupportedOperationException at java.util.Collections$UnmodifiableMap.remove(Collections.java:1345) at org.apache.commons.vfs2.provider.ftp.FtpFileObject.onChildrenChanged(FtpFileObject.java:271) at org.apache.commons.vfs2.provider.AbstractFileObject.childrenChanged(AbstractFileObject.java:240) at org.apache.commons.vfs2.provider.AbstractFileObject.notifyParent(AbstractFileObject.java:1931) at org.apache.commons.vfs2.provider.AbstractFileObject.handleCreate(AbstractFileObject.java:1577) at org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1866) at org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241) at org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184) ... I guess it is caused by the fact that children field is set to EMPTY_FTP_FILE_MAP at the moment onChildrenChanged() is invoked. I also do not like line 1866 in AbstractFileObject.java. To me it looks like it might be the real cause of the problem: FileObjectUtils.getAbstractFileObject(destFile).handleCreate(getType()); Must it not be destFile.getType()? But even if I am right about AbstractFileObject.java:1866, FtpFileObject.onChildrenChanged() must be corrected as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (VFS-558) java.lang.UnsupportedOperationException in FtpFileObject
L created VFS-558: - Summary: java.lang.UnsupportedOperationException in FtpFileObject Key: VFS-558 URL: https://issues.apache.org/jira/browse/VFS-558 Project: Commons VFS Issue Type: Bug Affects Versions: 2.0 Reporter: L I am getting the following exception in my code: java.lang.UnsupportedOperationException at java.util.Collections$UnmodifiableMap.remove(Collections.java:1345) at org.apache.commons.vfs2.provider.ftp.FtpFileObject.onChildrenChanged(FtpFileObject.java:271) at org.apache.commons.vfs2.provider.AbstractFileObject.childrenChanged(AbstractFileObject.java:240) at org.apache.commons.vfs2.provider.AbstractFileObject.notifyParent(AbstractFileObject.java:1931) at org.apache.commons.vfs2.provider.AbstractFileObject.handleCreate(AbstractFileObject.java:1577) at org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1866) at org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241) at org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184) ... I guess it is caused by the fact that children field is set to EMPTY_FTP_FILE_MAP at the moment onChildrenChanged() is invoked. I also do not like line 1866 in AbstractFileObject.java. To me it looks like it might be the real cause of the problem: FileObjectUtils.getAbstractFileObject(destFile).handleCreate(getType()); Must it not be destFile.getType()? But even if I am right about AbstractFileObject.java:1866, FtpFileObject.onChildrenChanged() must be corrected as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (VFS-558) java.lang.UnsupportedOperationException in FtpFileObject
[ https://issues.apache.org/jira/browse/VFS-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281942#comment-14281942 ] L edited comment on VFS-558 at 1/18/15 8:18 PM: Re: BTW2: do you use OnCallRefresher on purpose, I can imagine that makes FTP quite slow. Well, I do use CacheStrategy.ON_CALL, so yes, it is OnCallRefreshFileObject. My code has some checks before performing moveTo to verify the destination file is not there so I am not overwriting it. My tests gave me quite a lot of spurious warnings saying the file IS there before the rename while I quite sure it was not there. Adding the CacheStrategy.ON_CALL helped a lot but sometimes I still get false positives. was (Author: klv_m72): Re: BTW2: do you use OnCallRefresher on purpose, I can imagine that makes FTP quite slow. Well, I do use CacheStrategy.ON_CALL, so yes, it is OnCallRefreshFileObject. My code has some checks before performing moveTo to verify the destination file is not there so I am not overwriting it. My tests gave me quite a lot of spurious warnings saying the file IS there before the rename while I quite sure it was not there. Adding the CacheStrategy.ON_CALL helped a lot but sometimes I still get resulted in most of the false positive java.lang.UnsupportedOperationException in FtpFileObject Key: VFS-558 URL: https://issues.apache.org/jira/browse/VFS-558 Project: Commons VFS Issue Type: Bug Affects Versions: 2.0 Reporter: L I am getting the following exception in my code: java.lang.UnsupportedOperationException at java.util.Collections$UnmodifiableMap.remove(Collections.java:1345) at org.apache.commons.vfs2.provider.ftp.FtpFileObject.onChildrenChanged(FtpFileObject.java:271) at org.apache.commons.vfs2.provider.AbstractFileObject.childrenChanged(AbstractFileObject.java:240) at org.apache.commons.vfs2.provider.AbstractFileObject.notifyParent(AbstractFileObject.java:1931) at org.apache.commons.vfs2.provider.AbstractFileObject.handleCreate(AbstractFileObject.java:1577) at org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1866) at org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241) at org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184) ... I guess it is caused by the fact that children field is set to EMPTY_FTP_FILE_MAP at the moment onChildrenChanged() is invoked. I also do not like line 1866 in AbstractFileObject.java. To me it looks like it might be the real cause of the problem: FileObjectUtils.getAbstractFileObject(destFile).handleCreate(getType()); Must it not be destFile.getType()? But even if I am right about AbstractFileObject.java:1866, FtpFileObject.onChildrenChanged() must be corrected as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (VFS-558) java.lang.UnsupportedOperationException in FtpFileObject
[ https://issues.apache.org/jira/browse/VFS-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281944#comment-14281944 ] L commented on VFS-558: --- Re: BTW2: do you use OnCallRefresher on purpose, I can imagine that makes FTP quite slow. Well, I do use CacheStrategy.ON_CALL, so yes, it is OnCallRefreshFileObject. My code has some checks before performing moveTo to verify the destination file is not there so I am not overwriting it. My tests gave me quite a lot of spurious warnings saying the file IS there before the rename while I quite sure it was not there. Adding the CacheStrategy.ON_CALL helped a lot but sometimes I still get resulted in most of the false positive java.lang.UnsupportedOperationException in FtpFileObject Key: VFS-558 URL: https://issues.apache.org/jira/browse/VFS-558 Project: Commons VFS Issue Type: Bug Affects Versions: 2.0 Reporter: L I am getting the following exception in my code: java.lang.UnsupportedOperationException at java.util.Collections$UnmodifiableMap.remove(Collections.java:1345) at org.apache.commons.vfs2.provider.ftp.FtpFileObject.onChildrenChanged(FtpFileObject.java:271) at org.apache.commons.vfs2.provider.AbstractFileObject.childrenChanged(AbstractFileObject.java:240) at org.apache.commons.vfs2.provider.AbstractFileObject.notifyParent(AbstractFileObject.java:1931) at org.apache.commons.vfs2.provider.AbstractFileObject.handleCreate(AbstractFileObject.java:1577) at org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1866) at org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241) at org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184) ... I guess it is caused by the fact that children field is set to EMPTY_FTP_FILE_MAP at the moment onChildrenChanged() is invoked. I also do not like line 1866 in AbstractFileObject.java. To me it looks like it might be the real cause of the problem: FileObjectUtils.getAbstractFileObject(destFile).handleCreate(getType()); Must it not be destFile.getType()? But even if I am right about AbstractFileObject.java:1866, FtpFileObject.onChildrenChanged() must be corrected as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (VFS-558) java.lang.UnsupportedOperationException in FtpFileObject
[ https://issues.apache.org/jira/browse/VFS-558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L updated VFS-558: -- Comment: was deleted (was: Re: BTW2: do you use OnCallRefresher on purpose, I can imagine that makes FTP quite slow. Well, I do use CacheStrategy.ON_CALL, so yes, it is OnCallRefreshFileObject. My code has some checks before performing moveTo to verify the destination file is not there so I am not overwriting it. My tests gave me quite a lot of spurious warnings saying the file IS there before the rename while I quite sure it was not there. Adding the CacheStrategy.ON_CALL helped a lot but sometimes I still get resulted in most of the false positive ) java.lang.UnsupportedOperationException in FtpFileObject Key: VFS-558 URL: https://issues.apache.org/jira/browse/VFS-558 Project: Commons VFS Issue Type: Bug Affects Versions: 2.0 Reporter: L I am getting the following exception in my code: java.lang.UnsupportedOperationException at java.util.Collections$UnmodifiableMap.remove(Collections.java:1345) at org.apache.commons.vfs2.provider.ftp.FtpFileObject.onChildrenChanged(FtpFileObject.java:271) at org.apache.commons.vfs2.provider.AbstractFileObject.childrenChanged(AbstractFileObject.java:240) at org.apache.commons.vfs2.provider.AbstractFileObject.notifyParent(AbstractFileObject.java:1931) at org.apache.commons.vfs2.provider.AbstractFileObject.handleCreate(AbstractFileObject.java:1577) at org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1866) at org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241) at org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184) ... I guess it is caused by the fact that children field is set to EMPTY_FTP_FILE_MAP at the moment onChildrenChanged() is invoked. I also do not like line 1866 in AbstractFileObject.java. To me it looks like it might be the real cause of the problem: FileObjectUtils.getAbstractFileObject(destFile).handleCreate(getType()); Must it not be destFile.getType()? But even if I am right about AbstractFileObject.java:1866, FtpFileObject.onChildrenChanged() must be corrected as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (VFS-558) java.lang.UnsupportedOperationException in FtpFileObject
[ https://issues.apache.org/jira/browse/VFS-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281943#comment-14281943 ] L commented on VFS-558: --- Re: BTW2: do you use OnCallRefresher on purpose, I can imagine that makes FTP quite slow. Well, I do use CacheStrategy.ON_CALL, so yes, it is OnCallRefreshFileObject. My code has some checks before performing moveTo to verify the destination file is not there so I am not overwriting it. My tests gave me quite a lot of spurious warnings saying the file IS there before the rename while I quite sure it was not there. Adding the CacheStrategy.ON_CALL helped a lot but sometimes I still get resulted in most of the false positive java.lang.UnsupportedOperationException in FtpFileObject Key: VFS-558 URL: https://issues.apache.org/jira/browse/VFS-558 Project: Commons VFS Issue Type: Bug Affects Versions: 2.0 Reporter: L I am getting the following exception in my code: java.lang.UnsupportedOperationException at java.util.Collections$UnmodifiableMap.remove(Collections.java:1345) at org.apache.commons.vfs2.provider.ftp.FtpFileObject.onChildrenChanged(FtpFileObject.java:271) at org.apache.commons.vfs2.provider.AbstractFileObject.childrenChanged(AbstractFileObject.java:240) at org.apache.commons.vfs2.provider.AbstractFileObject.notifyParent(AbstractFileObject.java:1931) at org.apache.commons.vfs2.provider.AbstractFileObject.handleCreate(AbstractFileObject.java:1577) at org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1866) at org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241) at org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184) ... I guess it is caused by the fact that children field is set to EMPTY_FTP_FILE_MAP at the moment onChildrenChanged() is invoked. I also do not like line 1866 in AbstractFileObject.java. To me it looks like it might be the real cause of the problem: FileObjectUtils.getAbstractFileObject(destFile).handleCreate(getType()); Must it not be destFile.getType()? But even if I am right about AbstractFileObject.java:1866, FtpFileObject.onChildrenChanged() must be corrected as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (VFS-558) java.lang.UnsupportedOperationException in FtpFileObject
[ https://issues.apache.org/jira/browse/VFS-558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L updated VFS-558: -- Comment: was deleted (was: Re: BTW2: do you use OnCallRefresher on purpose, I can imagine that makes FTP quite slow. Well, I do use CacheStrategy.ON_CALL, so yes, it is OnCallRefreshFileObject. My code has some checks before performing moveTo to verify the destination file is not there so I am not overwriting it. My tests gave me quite a lot of spurious warnings saying the file IS there before the rename while I quite sure it was not there. Adding the CacheStrategy.ON_CALL helped a lot but sometimes I still get resulted in most of the false positive ) java.lang.UnsupportedOperationException in FtpFileObject Key: VFS-558 URL: https://issues.apache.org/jira/browse/VFS-558 Project: Commons VFS Issue Type: Bug Affects Versions: 2.0 Reporter: L I am getting the following exception in my code: java.lang.UnsupportedOperationException at java.util.Collections$UnmodifiableMap.remove(Collections.java:1345) at org.apache.commons.vfs2.provider.ftp.FtpFileObject.onChildrenChanged(FtpFileObject.java:271) at org.apache.commons.vfs2.provider.AbstractFileObject.childrenChanged(AbstractFileObject.java:240) at org.apache.commons.vfs2.provider.AbstractFileObject.notifyParent(AbstractFileObject.java:1931) at org.apache.commons.vfs2.provider.AbstractFileObject.handleCreate(AbstractFileObject.java:1577) at org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1866) at org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241) at org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184) ... I guess it is caused by the fact that children field is set to EMPTY_FTP_FILE_MAP at the moment onChildrenChanged() is invoked. I also do not like line 1866 in AbstractFileObject.java. To me it looks like it might be the real cause of the problem: FileObjectUtils.getAbstractFileObject(destFile).handleCreate(getType()); Must it not be destFile.getType()? But even if I am right about AbstractFileObject.java:1866, FtpFileObject.onChildrenChanged() must be corrected as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (VFS-558) java.lang.UnsupportedOperationException in FtpFileObject
[ https://issues.apache.org/jira/browse/VFS-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281942#comment-14281942 ] L commented on VFS-558: --- Re: BTW2: do you use OnCallRefresher on purpose, I can imagine that makes FTP quite slow. Well, I do use CacheStrategy.ON_CALL, so yes, it is OnCallRefreshFileObject. My code has some checks before performing moveTo to verify the destination file is not there so I am not overwriting it. My tests gave me quite a lot of spurious warnings saying the file IS there before the rename while I quite sure it was not there. Adding the CacheStrategy.ON_CALL helped a lot but sometimes I still get resulted in most of the false positive java.lang.UnsupportedOperationException in FtpFileObject Key: VFS-558 URL: https://issues.apache.org/jira/browse/VFS-558 Project: Commons VFS Issue Type: Bug Affects Versions: 2.0 Reporter: L I am getting the following exception in my code: java.lang.UnsupportedOperationException at java.util.Collections$UnmodifiableMap.remove(Collections.java:1345) at org.apache.commons.vfs2.provider.ftp.FtpFileObject.onChildrenChanged(FtpFileObject.java:271) at org.apache.commons.vfs2.provider.AbstractFileObject.childrenChanged(AbstractFileObject.java:240) at org.apache.commons.vfs2.provider.AbstractFileObject.notifyParent(AbstractFileObject.java:1931) at org.apache.commons.vfs2.provider.AbstractFileObject.handleCreate(AbstractFileObject.java:1577) at org.apache.commons.vfs2.provider.AbstractFileObject.moveTo(AbstractFileObject.java:1866) at org.apache.commons.vfs2.impl.DecoratedFileObject.moveTo(DecoratedFileObject.java:241) at org.apache.commons.vfs2.cache.OnCallRefreshFileObject.moveTo(OnCallRefreshFileObject.java:184) ... I guess it is caused by the fact that children field is set to EMPTY_FTP_FILE_MAP at the moment onChildrenChanged() is invoked. I also do not like line 1866 in AbstractFileObject.java. To me it looks like it might be the real cause of the problem: FileObjectUtils.getAbstractFileObject(destFile).handleCreate(getType()); Must it not be destFile.getType()? But even if I am right about AbstractFileObject.java:1866, FtpFileObject.onChildrenChanged() must be corrected as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DAEMON-235) When running java application via procrun (in jvm mode) an RMI thread triggers an EXCEPTION_ACCESS_VIOLATION
[
https://issues.apache.org/jira/browse/DAEMON-235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13418252#comment-13418252
]
J-L commented on DAEMON-235:
Just wanted to give a little update on this story.
This comment doesn't imply that procrun needs any fixing. Just posting for
people that might hit the same or if in the future similar problems where to
surface that do need fixing.
I 'fixed' the issue by having the JNI code running in a separate java process,
so the service runner kicking off another VM. That fixed everything for post
2008 (64bit) Windows OS. I only found out recently that for earlier versions of
Windows it didn't work.
Reason why it wasn't properly working still on those OS's was combination of
things. One the external process wasn't started with the -Xrs flag, which
caused problems when running as a service.
Second, in the child process I replaced the system input/output streams with
custom ones (as i used the stream to send control signals, but still allowing
other things to pass through). Note that the replacing of the streams wasn't
done when running within the same process, I'm mentioning it just because
running with procrun might have similar effects. With the replaced streams I
ran in to blockages caused by (or at least related to) the RMISecurity manager
(which is customized to allow more then the standard one, but still not overly
permissive).
I know it's these two things clashing as either not replacing the streams or
swapping the security manager to allow everything, made the thing work.
Eventually I choose the first solution (not in intercepting the streams in the
child process) as I had RMI to send control signals to the child process anyway.
Regards,
J-L
When running java application via procrun (in jvm mode) an RMI thread
triggers an EXCEPTION_ACCESS_VIOLATION
Key: DAEMON-235
URL: https://issues.apache.org/jira/browse/DAEMON-235
Project: Commons Daemon
Issue Type: Bug
Components: Procrun
Affects Versions: 1.0.5, 1.0.6, 1.0.7, 1.0.8
Environment: Windows XP SP3 and Win 7 SP1 x64
JDK 1.6.0_30 Server VM (same for client) 32 bit
Reporter: J-L
When running java application via procrun (in jvm mode) an RMI thread
triggers an EXCEPTION_ACCESS_VIOLATION
It happens with both with a series of VM tweaking options and no options at
all.
Using the same options executing from a bat file (with java.exe) does not
crash.
The crashing application is hosting an RMI registry at the moment of the
crash. An rmi call to the application is probably triggering the crash.
Adding an example hs_err_pid file. (Note: the java_home pointing elsewhere
doesn't effect the result).
{code:none}
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc005) at pc=0x7824f786, pid=5876,
tid=3656
#
# JRE version: 6.0_30-b12
# Java VM: Java HotSpot(TM) Server VM (20.5-b03 mixed mode windows-x86 )
# Problematic frame:
# C [MFC80.DLL+0x7f786]
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
--- T H R E A D ---
Current thread (0x20206c00): JavaThread RMI TCP
Connection(2)-192.168.56.101 daemon [_thread_in_native, id=3656,
stack(0x20ba,0x20c1)]
siginfo: ExceptionCode=0xc005, reading address 0x21301e90
Registers:
EAX=0x21301e74, EBX=0x22f83170, ECX=0x22f83170, EDX=0x
ESP=0x20c0f0c8, EBP=0x20c0f154, ESI=0x20c0f1c4, EDI=0x0004
EIP=0x7824f786, EFLAGS=0x00010202
Top of Stack: (sp=0x20c0f0c8)
0x20c0f0c8: 0002 d8a81ade 0004 20c0f1d4
0x20c0f0d8:
0x20c0f0e8: 20c0f110 206a
0x20c0f0f8: 00139f28
0x20c0f108: 0001 0027 20c0f06c 7712556e
0x20c0f118: 20c0f158 7c90e920 7c910060
0x20c0f128: 7c91005d 204cf1b7 206a
0x20c0f138: 22f83170 d92a86cf 0004 20c0f0cc
Instructions: (pc=0x7824f786)
0x7824f766: 78 ff 75 10 e8 70 98 fa ff 85 c0 59 59 75 0a b8
0x7824f776: 01 00 02 80 e9 39 04 00 00 ff 75 0c 8b 03 8b cb
0x7824f786: ff 50 1c 85 c0 75 0a b8 ff ff 00 80 e9 21 04 00
0x7824f796: 00 8d 7d c8 a5 a5 a5 a5 33 ff 39 7d d4 74 18 83
Register to memory mapping:
EAX=0x21301e74 is an unknown value
EBX=0x22f83170 is an unknown value
ECX=0x22f83170 is an unknown value
EDX=0x is an unknown value
ESP=0x20c0f0c8 is pointing into the stack for thread: 0x20206c00
EBP=0x20c0f154 is pointing into the stack for thread: 0x20206c00
