Dear Hoppists, Once in a blue moon the topic of authentication against Apache VFS protocols comes up again. The issues discussed focus on a number of different requirements that come up during discussions in various forms on the chat channels and elsewhere.
* It should be possible to use non-system level variables when configuring VFS features. This is because we don't expose any IVariables objects into the various HopVfs calls that resolve FileObjects, InputStream/OutputStream and so on. Important features should be configurable through a standard environment configuration file to match our lifecycle management vision. * We should be able to tie the authentication of VFS file systems against the Authentication Provider plugins which are already present in Apache Hop but go unused. For example if we have a secure FTP target and we want to authenticate using a secure key we should be able to provide that key and use that to authenticate against a certain hostname. * It should be possible to authenticate differently to several systems in the same project or environment. For example it should be possible to authenticate to different hdfs:// servers at the same time or use multiple Amazon s3:// accounts at the same time. This would allow us to transfer data from one account to another for example. * It should be possible to create symbolic links through our VFS driver usage where we map one folder to another hiding the implementation details. See also: https://issues.apache.org/jira/browse/HOP-3795 Architecturally I think that what this means is that we first need to create Authentication metadata elements which use the Authentication Provider plugins. https://issues.apache.org/jira/browse/HOP-3796 After that we can use those authentication provider metadata elements to make it super easy to configure VFS authentication simply by pointing to whichever method we want to use. We can do that in Hop VFS Authentication metadata elements. Technically what this means is that almost 1000 method uses of class HopVfs need to be fed the appropriate IVariables and IHopMetadataProvider instance at the location where the method is used (the lowest level possible). This sounds like a lot of work but I feel like that we should get this out of the way sooner rather than later since the topic just keeps coming back. The idea that nobody else out there is doing even remotely like it is not a deterrent, it's an incentive :-) I would love to hear your ideas and check if anyone else had any requirements that we should take into account. Thanks in advance, Matt
