Re: [vfs] FileSystemManager construction rethought
Rami Ojares wrote:
Full acceptance with satisfaction guaranteed.
DelegatingFileSystemOptionsBuilder is our friend.
FileSystemOptions fso = new FileSystemOptions();
DelegatingFileSystemOptionsBuilder delegate = new
DelegatingFileSystemOptionsBuilder(VFS.getManager());
delegate.setConfigString(fso, sftp, identities, c:/tmp/test.ident);
delegate.setConfigString(fso, http, proxyPort, 8080);
delegate.setConfigClass(fso, sftp, userinfo,
TrustEveryoneUserInfo.class);
final String[] identityPaths = new String[]
{
/file1,
/file2,
};
delgate.setConfigStrings(opts, sftp, identities, identityPaths);
The setConfigString is used to set the option by converting the string
to the needet setter parameter type.
This is done by searching a string constructor, or a static
valueOf(String) method on the requied type.
The setConfigClass is used to pass in a class-name. A new instance of
this class is created by invoking the no-arg constructor. This instance
is passed to the setter - setUserInfo(UserInfo) in the above case.
The setConfigStrings is an example tho show the use of arrays. All
elements within the array are converted to File objects as the method
signature of SftpFileSystemConfigBuilder.setIdentites is
setIdentites(.., File[])
You can find the possible keys (proxyPort, ...) within the various
*ConfigBuilder classes by searching all set* methods.
The keys are case insensitive (they are always converted to lower case).
--
Mario
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [vfs] FileSystemManager construction rethought
Hi Mario, Here I have listed my current understanding of VFS in connection with the configuration issues. Could you please correct any mistakes you find. - Manager contains FileProviders - Manager supports all schemas for which it has configured a provider - FileProviders create a FileSystem the first time a file is resolved against them - FileProviders create a new FileSystem everytime a new FileSystemOptions is encountered - FileSystem is in a way active meaning that it contains a live connection over some protocol if that is needed to access the files on the filesystem. - User can create layered filesystems on top of originating filesystems - User can create virtual filesystems and add junctions to them So if these points are correct the configuration data stucture contains the following: - Manager specific configuration independent of providers Q: What are these? - Provider specific default configuration (manager instantiates and configures the providers it supports) - ad hoc configuration when resolveFile is called And then my understanding of the current configuration mechanism - StandardFileSystemManager configures the manager and providers from xml - resolveFile method provides FileSystemOptions that results in creation of FileSystem if the options were never given before. - every invocation of resolveFile can potentially result in creation of a FileSystem - FileSystemConfigBuilders provide type safety to FileSystemOptions - FileProvider must provide a FileSystemConfigBuilder that is used to create FileSystemOptions suitable for that provider's filesystems == FileSystemOptions are the only configuration data structure in place at the moment and it is the duty of FileSystemManager and FileProvider implementations to provide for their own configuration. Let's consider configuration of a logger to FileSystemManager. setLogger() is a method in DefaultFileSystemManager. Further it utilizes setLogger method in VfsComponent interface that is implemented by FileProviders and FileSystems (among others). So from the point of view of ant task that wants to set a logger to the manager it uses, there should be setLogger() in FileSystemManager interface. Because if it can choose manager implementation then it would have to use introspection to find setLogger() and it would have no guarentee that such method exists in the implementation. This might not be seen as problem. But let's take it further. Consider ant fragment vfs-manager refid=mymanager class=MyManager configuration=path/to/conf/ vfs-copy todir=sftp://host/usr/local/var; vfs-manager ref=mymanager/ /vfs-copy In the configuration file there would already be pointers to knownHosts and privateKey files so one would not have to tell them inside the uri. But if one would tell them for example sftp://id=/path/to/.private.key;knownHosts=some/[EMAIL PROTECTED]/usr/local/var then that would just override the providers defaults set from the configuration file. Now the configuration (file) can be passed as path, File, FileObject, DOM or whatever. But it would be nice if FileSystemManager interface would dictate at least some kind of configuration framework so the ant tasks would not have to mention any implementations. - Rami Ojares - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [vfs] FileSystemManager construction rethought
Now the configuration (file) can be passed as path, File, FileObject, DOM or whatever. Here the word Now does not mean that now the implementation is ... It means In general it is possible ... - rami - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [vfs] FileSystemManager construction rethought
Rami Ojares wrote: All you have written seems right - hard work ;-) But if one would tell them for example sftp://id=/path/to/.private.key;knownHosts=some/[EMAIL PROTECTED]/usr/local/var then that would just override the providers defaults set from the configuration file. But i have to say i dont like to add all those configuration stuff to the uri - maybe this might be a handy place, but adding e.g. path elements like those in your example might be a real pain. It might be really hard to construct a correct uri. Now the configuration (file) can be passed as path, File, FileObject, DOM or whatever. But it would be nice if FileSystemManager interface would dictate at least some kind of configuration framework so the ant tasks would not have to mention any implementations. I can second this too, and i see not discrepance to the current system. vfs-manager refid=mymanager class=MyManager configuration=path/to/conf/ vfs-options id=options1 vfs-option scheme=http name=proxyHost value=yourProxyHost / vfs-option scheme=sftp name=userInfo className=yourUserInfoImplementation / vfs-option scheme=sftp name=knownHosts value=/path/to / vfs-option scheme=sftp name=identities valuepath/to/ident1/value valuepath/to/ident2/value /vfs-options /vfs-options vfs-options id=options2 vfs-option ... some other configuration ... / /vfs-options Now there could be a class e.g. GeneralFileSystemConfigBuilder with (vfs-options name= value=) setConfigValue(String scheme, String name, String value) (vfs-options name= value/value) setConfigValue(String scheme, String name, String[] values) (vfs-options name= className=) setConfigClass(String scheme, String name, String className) Which tries to locate the correct *FileSystemConfigBuilder, tries to convert the given value/class to the expected parameter and calls the method. All this can be done using reflection and a configuration mistake is fastly shown. I will implement this class if we find a consent. Now there is the problem how to connect the options to a given uri. Maybe, if we do not connect them to the uri we could use a construct like this. vfs-copy todir=sftp://host/usr/local/var; vfs-manager ref=mymanager/ vfs-options ref=options1/ /vfs-copy Or i can think about an easy implementation like this: vfs-copy todir=[EMAIL PROTECTED]://host/usr/local/var vfs-manager ref=mymanager/ /vfs-copy Then the ant task should strip the @options1 and lookup a table to find the FileSystemOptions for resolvFile() To do the same with the FileSystemManager - as you have said - we have to create a separate data structure first. I will have a look at it, but for now we can stick on the xml file. Might this find your acceptance? -- Mario - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [vfs] FileSystemManager construction rethought
But i have to say i dont like to add all those configuration stuff to the uri - maybe this might be a handy place, but adding e.g. path elements like those in your example might be a real pain. It might be really hard to construct a correct uri. I agree. It was just an example. Now there could be a class e.g. GeneralFileSystemConfigBuilder with (vfs-options name= value=) setConfigValue(String scheme, String name, String value) (vfs-options name= value/value) setConfigValue(String scheme, String name, String[] values) (vfs-options name= className=) setConfigClass(String scheme, String name, String className) Which tries to locate the correct *FileSystemConfigBuilder, tries to convert the given value/class to the expected parameter and calls the method. All this can be done using reflection and a configuration mistake is fastly shown. I will implement this class if we find a consent. Nice. Very nice. Now there is the problem how to connect the options to a given uri. Ant can always call resolveFile(String name, FileSystemOptions fileSystemOptions) to resolve files if the task or datatype has been provided with some options. vfs-copy todir=[EMAIL PROTECTED]://host/usr/local/var vfs-manager ref=mymanager/ /vfs-copy Then the ant task should strip the @options1 and lookup a table to find the FileSystemOptions for resolvFile() Maybe not. :) To do the same with the FileSystemManager - as you have said - we have to create a separate data structure first. I will have a look at it, but for now we can stick on the xml file. The addition that I was thinking would be minor. Keep XML configuration the way it is but add configure(File/FileObject/InputStream) to FileSystemManager interface. And document the syntax of the xml file that is used to configure manager and providers. Might this find your acceptance? Full acceptance with satisfaction guaranteed. - rami - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [vfs] FileSystemManager construction rethought
Hello Rami, here are some thoughts about configuration: *) GlobalConfiguration: As you have seen, the StandardFileSystemManager currently uses an xml file to configure the system. I tried to replace the parsing of the xml file by commons-digester beanutils but failed as it comes to the point to add those dependencies to vfs. Other people vetoed against it (Too large jars) If i had ever finished this new configuration parsing the result would be a object structure useable to pass to the DefaultFileSystemManager. The GlobalConfiguration is one of the relicts of this try and maybe i will remove it and place its current members to the DefaultFileSystemManager. *) FileSystemConfigBuilder: Is the try to configure various aspects of a FileSystem (e.g. for http proxy authentication, for sftp the key location, ...) So the container for all those options is a instance of FileSystemOptions, but there are no public methods to add configuration parameters into it. Every filesystem which could be configured do have its own FileSystemConfigBuild e.g. for sftp this is SftpFileSystemConfigBuilder. Now the idea behind this construct was to have a typesafe configuration framework. FileSystemOptions opts = new FileSystemOptions(); SftpFileSystemConfigBuilder.getInstance().setKnownHosts(opts, new File(path-to-your-known_hosts-file)); HttpFileSystemConfigBuilder.getInstance().setProxyHost(ops, proxy.domain.org); On resolveFile it is possible to pass this opts in and if it comes to the sftp-filesystem it picks its options out of opts. Currently this is the only way to pass options into the system, but for sure, we need a method on the DefaultFileSystemManager to set the defaults and allow some kind of override of this options at resolveFile(). Conclusion: IMHO you could finish your work without loose your time with handling of FileSystemOptions - this is a complicate part on its own, to make this work in ant and from the xml configuration we need reflection. Drop your FileSystemConfiguration and replace it by a simple string which holds the class for the filesystem manager. I will have a look at GlobalConfiguration (and maybe drop it too) and find some way how to pass in a global/defaults/... FileSystemOptions object. One question that arose is that why would someone want to write a different FileSystemManager implementation. Could the DefaultFileSystemManager be made so configurable that this would not be needed? One reason could be not to use the xml file for configuration, but write a custom FileSystemManager which does the whole configuration in code. -- Mario - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [vfs] FileSystemManager construction rethought
Mario Ivankovits wrote on Wednesday, June 23, 2004 9:48 AM: Hello Rami, here are some thoughts about configuration: *) GlobalConfiguration: As you have seen, the StandardFileSystemManager currently uses an xml file to configure the system. I tried to replace the parsing of the xml file by commons-digester beanutils but failed as it comes to the point to add those dependencies to vfs. Other people vetoed against it (Too large jars) If i had ever finished this new configuration parsing the result would be a object structure useable to pass to the DefaultFileSystemManager. The GlobalConfiguration is one of the relicts of this try and maybe i will remove it and place its current members to the DefaultFileSystemManager. Why not use a lightweight XML-Java mapper like XStream or directly xpp3 ? -- Jörg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [vfs] FileSystemManager construction rethought
I tried to replace the parsing of the xml file by commons-digester beanutils but failed as it comes to the point to add those dependencies to vfs. Other people vetoed against it (Too large jars) That is a shame. But isn't it so that we should first come up with a configuration structure (data structure and configuration policy) for vfs independent of where we get the actual data to this data structure. Since it takes a lot of data to configure FileSystemManager it is obvious that we need xml file to hold this data. And the current xml structure is a good start. Now the idea behind this construct was to have a typesafe configuration framework. The configuration container could IMHO be not type safe. Because once you call configure() right after instantiation the filesystem manager passes provider specific configuration to providers and all the providers would configure themselves too. And any missing or erroneous stuff would be reported at that point. Which to me would be a very natural point for letting the user know that something is wrong in his configuration file. The configuration data structure could mention the providers and their data types making it type safe. Anyway, now it seems to me overly complex when the issue is quite simple. In the simplest case we would just need a naming convention and meaning for different configuration elements. Example: Sftp provider accepts the following configuration elements - sftp.known-hosts A path (String) that must point to a valid known-hosts file - sftp.private-key A path (String) that must point to a valid private key file in XXX encoding ... This same configuration data structure class could be used when overriding in case of resolveFile() One question that arose is that why would someone want to write a different FileSystemManager implementation. Could the DefaultFileSystemManager be made so configurable that this would not be needed? One reason could be not to use the xml file for configuration, but write a custom FileSystemManager which does the whole configuration in code. Is this really reason enough for the added complexity? - rami - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [vfs] FileSystemManager construction rethought
Here are some small changes that I came across when making the changes compilable.
I changed configure method to init because there was already
private configure(String)
method in DefaultFileSystemManager but the existing init method
was without arguments.
I saw also that there is quite a lot of configuration code in place
(GlobalConfiguration class and the configure method mentioned above
that parses configuration from XML). But I did not see the overall idea
so you'll have to explain it to me Mario. But it all seems somehow
still coupled with the DefaultFileSystemManager.
One question that arose is that why would someone want to write
a different FileSystemManager implementation. Could the DefaultFileSystemManager
be made so configurable that this would not be needed?
Anyways, here is a compilable listing of the code I sent in the previous email.
- rami
* VfsTask.java *
package org.apache.commons.vfs.ant;
import org.apache.commons.vfs.*;
import org.apache.tools.ant.*;
/**
* Super class of Vfs Ant tasks that takes care of FileSystemManager handling.
*/
public class VfsTask extends Task {
private FileSystemManager manager;
private AntLogger logger;
protected FileSystemManager getManager() {
if (manager == null)
setManager(null);
return manager;
}
protected void setManager(FileSystemConfiguration conf) {
if (conf == null) {
conf = new FileSystemConfiguration(
org.apache.commons.vfs.impl.StandardFileSystemManager
);
}
// put AntLogger always as logger
conf.setParameter(logger, getLogger());
this.manager = AntHelper.getManager(this, conf);
}
protected AntLogger getLogger() {
if (logger == null)
logger = new AntLogger(this);
return logger;
}
/**
* Resolves a URI to a file, relative to the project's base directory.
*
* @param uri The URI to resolve.
*/
protected FileObject resolveFile(final String uri) {
try {
return getManager().resolveFile(getProject().getBaseDir(), uri);
}
catch (FileSystemException fse) {
throw new BuildException(fse);
}
}
}
* VfsDataType.java *
package org.apache.commons.vfs.ant;
import org.apache.commons.vfs.*;
import org.apache.tools.ant.types.*;
import org.apache.tools.ant.BuildException;
/**
* Super class of Vfs Ant data types that takes care of FileSystemManager
* handling.
*/
public class VfsDataType extends DataType {
private FileSystemManager manager;
private AntLogger logger;
protected FileSystemManager getManager() {
if (manager == null)
setManager(null);
return manager;
}
protected void setManager(FileSystemConfiguration conf) {
if (conf == null) {
conf = new FileSystemConfiguration(
org.apache.commons.vfs.impl.StandardFileSystemManager
);
}
// put AntLogger always as logger
conf.setParameter(logger, getLogger());
this.manager = AntHelper.getManager(this, conf);
}
protected AntLogger getLogger() {
if (logger == null)
logger = new AntLogger(this);
return logger;
}
/**
* Resolves a URI to a file, relative to the project's base directory.
*
* @param uri The URI to resolve.
*/
protected FileObject resolveFile(final String uri) {
try {
return getManager().resolveFile(getProject().getBaseDir(), uri);
}
catch (FileSystemException fse) {
throw new BuildException(fse);
}
}
}
* AntHelper.java *
package org.apache.commons.vfs.ant;
import org.apache.commons.vfs.*;
import org.apache.tools.ant.*;
import java.util.*;
/**
* Holds a map of FileSystemKey - FileSystemManger
* When a Task or DataType asks for FileSystemManger, AntHelper looks it up
* in the map. If it does not exist, it is created, added to the map and a
* BuildListener is added to the project so that when the project finishes it
* closes the FileSystemManger and removes it from the map.
*/
public class AntHelper {
private static Map managers = new HashMap();
public static FileSystemManager getManager(ProjectComponent projectComponent,
FileSystemConfiguration conf) {
FileSystemKey key = new FileSystemKey(projectComponent.getProject(), conf);
FileSystemManager manager = (FileSystemManager) managers.get(key);
if (manager == null) {
try {
manager = VFS.createManager(conf);
}
catch (FileSystemException fse) {
throw new BuildException(fse);
}
projectComponent.getProject().addBuildListener(new CloseListener(key));
