I am not sure you are correct in your responses below. The only proof that I am wrong is providing the code to do this.
1) How do you specify a regex to download files with todays date that will run in synapse unattended for weeks ? Yes I can hard code an expression to match 31 March 2009 and after but that is no good today, is it. Java has the concept of Date NOW; this is what I mean by today. You say this is "Quite easy" please show me how, as I cannot see how this can be done ? If it is easy then my problem is solved. Great, but this is my question ? 2) If an SFTP server has 2000 old files permanently on it now that will not be removed; thats grows every day. How to I download just the new files ? E.G Those placed on there with todays date in the name. I have no permission on the partners SFTP server to MOVE files or DELETE them, they just stay there. It is not a good assumption that files can be MOVED or DELETED on another organisations SFTP server. Very rarely in real B2B do you get this luxury. 3) You suggest SYnapse has a concept of new files ? How does this work ? 4) Are you saying that when I start Synapse with an SFTP mediator it remembers every file downloaded ever in the past; so that when we re-start Synapse (after maintenence) it remembers the old files. If this is true this is great. I am interested to know where this record of files is kept. Where is the file ? 5) Otherwise I am missing some very interesting behaviour of the Synapse VFS Mediator that does not seem to be provided by the samples or the documentation or any prior Forum discussions. I am quite happy to raise a Jira but there is no need if Synapse can do this, easily, as you say ? But at this point only the code will help me understand this. I do not understand this comment about being in a hurry; we are evaluating Synapse to solve a commercial problem and unfortunately we have encountered many issues that we are trying to resolve. We chose a particular set of use cases to solve and they were fixed months ago. This is called an "architectural prototype". It is used to mitigate the risk of a new product or type of solution/technology in a project. Unfortunately when I think all the hurdles are solved; then we stumble over new ones. Yes unfortunately real world projects have definite requirements and budgets and deadlines. I do not see the point of open source software if it cannot solve commercial problems; unless Synapse is just an Academic Research project ? I am sure WS02 are using Synapse to solve real world commercial problems and not just for research. I am sure they make money from using the code. Unfortunately not all Synapse users are expert enough to contribute yet; so once again I mention the Catch 22 of your comments. It takes time for new people to get up to speed so they can contribute. So I spent about 4 hours researching this today before posting; and I believe this is a contribution. We have real world requirements that would improve Synapse; this is a contribution. Asking questions that do not seem to be covered by documentation; so that others can benefit is a contribution. Thats all I can offer at the moment. Unfortunately we have made an extensive investment in Synapse, in time, and at this point I am trying not to have to drop the use of this product and trying my hardest to demonstrate that this product can solve just one single real world problem for us. Am I in a hurry, yes, we have passed the deadline. Asankha C. Perera wrote: > > Hi Kim >> We need to download files created only today on a SFTP server. Initially >> I >> thought the regular expression parameter could be used to do this, but >> now I >> don't think its possible. >> >> A further major design fault of the VFS mediator is that I don't think >> these >> parameters can be programatically set. > This is not a design fault, but the way it was designed.. To Synapse, > VFS is just a transport - underneath Axis2.. which makes things a bit > distant. Ofcourse it may not be exactly what you want, but has use for > some other common scenarios.. Synapse is expected to poll locations for > new files.. you can poll directories and also provide a file name pattern. >> Requirement: The SFTP server we need to connect to keeps every file the >> partner creates, on it for months. We only need files with todays date in >> the name. Unfortunately we cannot change the partners policy here or hwo >> they name files. >> >> The file naming scheme is: >> CODEMMDDYYNN.TYPE >> Where CODE is a set parameter >> Where TYPE is a set parameter >> Where NN is the increment number for files created within the day. So a * >> here. >> The rest is the date. >> >> So the server may have 2000 files and 3 created today. >> How can I download just those 3 ? >> >> Originally I thought I could create the regular expresssion parameter in >> Java code and pass to VFS. >> But I don't think this is possible ? >> > Not to poll, but you can do this to write - i.e. dynamically decide the > path to write. To poll, you should still be able to specify the regex > such that MMDDYY matches any dates after today, quite easily.. >> VFS parameter are isolated from Synapse. Never thought Synapse would be >> so >> inflexible. >> >> Only solution left is to do it completely in a regular expression, not >> sure >> this is feasible ? >> Can a Java regular expression know the date today ? >> Is this compatible with how VFS uses the expression ? >> >> Is there any other way to set the VFS parameters ? >> >> Alternative is to fix the way VFS parameters are set, e.g via a property >> rather than inflexible text only. (However, if VFS is a proxy this cannot >> be >> done either, because parameter cannot get set prior to proxy call) >> OR >> call VFS from Java - but then why use Synapse ? >> >> A final solution is to use a Java program to write the synapse.xml file, >> with the required text. Then hot-load the synapse.xml every midnight. >> However, again, may as well just stick with Java solution. >> >> This is about the last nail in the coffin for our problems with getting >> Synapse to solve a simple mediation problem. >> > Seems like using pure Java might be a good workaround for your case for > now, since you seem to be in a hurry, and all developers of Synapse are > volunteers and thus we cannot guarantee any time frames for > enhancements. We would be happy if you could open JIRA's though so that > someone could attend to them when they have time. > > cheers > asankha > > -- > Asankha C. Perera > AdroitLogic, http://adroitlogic.org > > http://esbmagic.blogspot.com > > > > > > -- View this message in context: http://www.nabble.com/Can-VFS-SFTP-be-used-to-download-todays-files-only-tp22901452p22903903.html Sent from the Synapse - User mailing list archive at Nabble.com.
