Re: [VFS] FileSystem close
Mail from Johan - Forwarded to the mailinglist too Mario On Wed, 07 Jul 2004 22:20:12 +0200, Mario Ivankovits [EMAIL PROTECTED] wrote: So i picked up the SoftRefFilesCache and adapted it slightly. The idea behind is - as long as a FileObject is strongly reachable by the jvm there must be someone who is willing to access the file and thus it is not allowed to close the filesystem. As soon as the last file of a certain filesystem is garbage-collected the filesystem can be (and will be) closed. For sure - now you do not have that control over this release process, but it should work well. Much better idea - a small problem may be that garbage collection can not be forced and resources could potentially build up substantially before they are cleared ... One has simply to ensure to release all references (assign null to it if it is a global member) to make this work. @Johan: I would like to remove the useCount (freeUnusedResources) stuff from vfs again. Could you please try this in your environment. Instantiate your vfs manager like this: StandardFileSystemManager manager = new StandardFileSystemManager(); manager.setFilesCache(new SoftRefFilesCache()); manager.init(); and remove the call to freeUnusedResources - and - as long as you do not use any input/output stream - the FileObject.close() is also no longer needet. Have tried this and it's working fine - had to suggest GC a couple of times, but the references (and therefore the session) disappear after a while - excellent :) Cheers, Johan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [VFS] FileSystem close
Johan Lindquist wrote: So, something like this: FileSystemOptions fso = new FileSystemOptions(); SftpFileSystemConfigBuilder.getInstance().setUserInfo(fso, new TrustEveryoneUserInfo()); FileObject inFile = fsManager.resolveFile(sftp://someurl;, fso); // do something with the file fs = inFile.getFileSystem(); fs.close(); What are the general thoughts about this? First, there is another workaround for the close problem in multithreaded environments if you create your own Manager (like VFS.getManager() does) but attach it to a threadLocal. I have done this in our wep-application and use a servlet filter to ensure the manager is closed after the request. I know, also only a workaround, but a thread-safe one ;-) However, i see the problem and i would like to solve it in a clear way. One solution could be to force the developer to close EVERY resolved file, that way we could maintain a use-count on the file-system and if this drop to zero a) automatically or b) by calling cleanup() on the manager close this filesystem. Exposing the filesystem close() is not a solution, if you use a global manager (= one instance for all threads) you cant be sure no other thread will use the same filesystem and thus if you call close, it will unexpectedly closed for the other thread too. I tend to implement this use-count, but we have to close EVERY file, even if we only used it to get e.g. the file-type. What do you think? -- Mario - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [VFS] FileSystem close
Rami Ojares wrote: Hi, I asked about this a little while ago - when you resolve a file for a SFTP url, the SFTP file system maintains the session until the file manager is closed - the solution you propsed (see below) works well, but this will not work if I try to use this in a multi-threaded environment. What are the general thoughts about this? One possible direction to take would be to develop the pooling mechanism (maybe use commons-pooling?) and give an option to the sftp provider pooling=true|false. Disabling pooling (even the one-sized) is IMHO not an option as this might make this FileSystem dog slow and the network bandwith might greatly increase - for every request we have to run through the authentication process and reattach the sftp channel. -- Mario
Re: [VFS] FileSystem close
On Sun, 04 Jul 2004 08:13:24 +0200, Mario Ivankovits [EMAIL PROTECTED] wrote: Johan Lindquist wrote: So, something like this: FileSystemOptions fso = new FileSystemOptions(); SftpFileSystemConfigBuilder.getInstance().setUserInfo(fso, new TrustEveryoneUserInfo()); FileObject inFile = fsManager.resolveFile(sftp://someurl;, fso); // do something with the file fs = inFile.getFileSystem(); fs.close(); What are the general thoughts about this? First, there is another workaround for the close problem in multithreaded environments if you create your own Manager (like VFS.getManager() does) but attach it to a threadLocal. I have done this in our wep-application and use a servlet filter to ensure the manager is closed after the request. I know, also only a workaround, but a thread-safe one ;-) However, i see the problem and i would like to solve it in a clear way. One solution could be to force the developer to close EVERY resolved file, that way we could maintain a use-count on the file-system and if this drop to zero a) automatically or b) by calling cleanup() on the manager close this filesystem. Exposing the filesystem close() is not a solution, if you use a global manager (= one instance for all threads) you cant be sure no other thread will use the same filesystem and thus if you call close, it will unexpectedly closed for the other thread too. I tend to implement this use-count, but we have to close EVERY file, even if we only used it to get e.g. the file-type. What do you think? Closing every file could be painful - at least to remember - but I think it would guarantee (more) that resources are released as required. Streams already require this and developers need to be careful with resources in general, so it is just a matter of education ;) Realise now how the filesystem close would not work in multi-threaded environments either - unless it was made conditionally (using the counter you describe above) but then the cleanup method is definately better ... Will fiddle with the thread-local work-around for the moment but it seems the use-count option is the way to proceed. I can have a look at it (time allowing) if you want some more feedback? Thanks, Johan -- Mario - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- you too? - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[VFS] FileSystem close
Hi, I asked about this a little while ago - when you resolve a file for a SFTP url, the SFTP file system maintains the session until the file manager is closed - the solution you propsed (see below) works well, but this will not work if I try to use this in a multi-threaded environment. FileSystemOptions fso = new FileSystemOptions(); SftpFileSystemConfigBuilder.getInstance().setUserInfo(fso, new TrustEveryoneUserInfo()); FileObject inFile = fsManager.resolveFile(sftp://someurl;, fso); // do something with the file StandardFileSystemManager fsm = (StandardFileSystemManager) VFS.getManager(); fsm.close(); A step closer (if i understand the arch. correctly) would be to expose the close method (as available on the SFTPFileSystem class for example) in the FileSystem interface. This allow you to retrieve the file system for the file object being accessed and manually close it. It is not the ideal solution, but would free the jsch session being held in the background. Does this have any side-effects i haven't thought about - one i can think of is (again) concurrent access to the same filesystem. So, maybe the file system could close itself only if there are no more references to files within it? So, something like this: FileSystemOptions fso = new FileSystemOptions(); SftpFileSystemConfigBuilder.getInstance().setUserInfo(fso, new TrustEveryoneUserInfo()); FileObject inFile = fsManager.resolveFile(sftp://someurl;, fso); // do something with the file fs = inFile.getFileSystem(); fs.close(); What are the general thoughts about this? Regards, Johan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [VFS] FileSystem close
Hi, I asked about this a little while ago - when you resolve a file for a SFTP url, the SFTP file system maintains the session until the file manager is closed - the solution you propsed (see below) works well, but this will not work if I try to use this in a multi-threaded environment. What are the general thoughts about this? One sftp filesystem keeps at most one idle connection open. So it is like connection pool whose max size is one. Now you have multiple threads accessing the same sftp server but you would not like to have an idle connection open, right? One possible direction to take would be to develop the pooling mechanism (maybe use commons-pooling?) and give an option to the sftp provider pooling=true|false. Would this solve your problem? - rami - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
