Sorry, I seemed to miss the exception that I am getting: Caused by: java.lang.IllegalArgumentException: Could not find a suitable setter for property: filter as there isn't a setter method with same type: java.lang.String nor type conversion possible: No type converter available to convert from type: java.lang.String to the required type: org.apache.camel.component.file.GenericFileFilter with value #fileExistsFilter
Not sure why but it seems that the filter is not getting the param from the registry but trying to pass the value in as a normal string. Based on the last comment it sounds like this should be working? On Thu, Jun 28, 2012 at 11:21 PM, Claus Ibsen <claus.ib...@gmail.com> wrote: > On Fri, Jun 29, 2012 at 12:32 AM, David Capwell <dcapw...@gmail.com> > wrote: > > Playing with the delay fields defined in file2 seems to improve download > > speeds. Now that this is working better I want to be able to skip any > file > > that has already been processed (should persist on restart). It looked > to > > me that I should be able to set this up by using the filter option, but I > > am having a hard time figuring out how to use this option without using > > spring. > > > > Yeah you can either use the filter or the idempotent consumer. The > latter is the EIP pattern that fits this problem. > The file/ftp component has baked in of both. And the latter has > plugins for various persistent storages. > > The code below seems okay, if you configure the endpoint from the uri, > then you need to refer to your > bean that implements the filter using the # notation, as you do. > > Then you need to enlist that bean in the registry. In Spring XML that > happens automatic when you do a <bean>. > When you use pure Java, you need to pass in a registry implementation > when you create the camel context, > as you do with the code below using the SimpleRegistry. > > Then its just a matter of putting the bean in the simple registry, as > shown in the code. > > > The unit test of the camel-ftp component has plenty of examples as well > > https://svn.apache.org/repos/asf/camel/trunk/components/camel-ftp/src/test/java/ > > > > here is the code I am using (might not compile cause its copy/paste from > > diff places): > > > > SimpleRegistry registry = new SimpleRegistry(); > > registry.put("fileExistsFilter", new CopyFilter()); > > > > CamelContext context = new DefaultCamelContext(registry); > > context.addRoutes(new RouteBuilder() { > > @Override > > public void configure() throws Exception { > > from("sftp://example.com/path/to/file?password="; + pass + > > "&binary=true&delay="+delay+ "&initialDelay=" + delay + > > "&filter=#fileExistsFilter") > > .to("file://data/ftp?fileExist=Ignore"); > > > > } > > }); > > context.start(); > > > > ... > > public static class CopyFilter implements GenericFileFilter { > > > > @Override > > public boolean accept(GenericFile genericFile) { > > // check to see if file exists locally > > File localFile = new File("data/ftp", genericFile.getFileName()); > > if(localFile.exists()) { > > // check to make sure the sizes match > > if(localFile.length() == genericFile.getFileLength()) { > > // file has already been copied, ignore > > return false; > > } > > } > > return true; > > } > > } > > > > > > Thanks for your time reading this email > > > > On Thu, Jun 28, 2012 at 6:07 AM, David Capwell <dcapw...@gmail.com> > wrote: > > > >> I'll take a look at file2 to see if that helps. > >> > >> My goal is to push the data from FTP to s3. Copying locally so I know > >> what has been processed, > >> On Jun 28, 2012 12:48 AM, "Claus Ibsen" <claus.ib...@gmail.com> wrote: > >> > >>> Hi > >>> > >>> As mentioned on the ftp2 page > >>> http://camel.apache.org/ftp2 > >>> > >>> See the file2 page for more options as the ftp component inherits these > >>> options > >>> http://camel.apache.org/file2.html > >>> > >>> The delay and consumer.delay option is the same. delay is just > >>> shorthand for consumer.delay. > >>> > >>> And no you cannot avoid the download if you later want to upload it > >>> someplace else, and the file exists. > >>> The ftp consumer don't know about this. > >>> > >>> A tricky improvement could be to add a new option to only download the > >>> file on demand, but that is a bit tricky > >>> as you would need a live connection and the ftp client to still be > >>> around and connected. > >>> > >>> > >>> What are you trying to do? To sync files between 2 ftp servers? > >>> > >>> > >>> > >>> On Thu, Jun 28, 2012 at 2:15 AM, David Capwell <dcapw...@gmail.com> > >>> wrote: > >>> > I am trying to use sftp (camel 2.9.2) and notice that every download > is > >>> > spread out to one every 10 seconds (files are around 10kb). Based > off > >>> the > >>> > examples in http://camel.apache.org/ftp2.html it looks like there > are > >>> delay > >>> > options (they are not listened under options, but used in example) > but > >>> when > >>> > I use them nothing changes. How can I have the client pull files > >>> faster? > >>> > > >>> > Another question, if I am trying to write the ftp files locally, is > >>> there > >>> > any way to avoid the file download if the local file is around? > >>> > I append fileExist=Ignore in the to() but it seems that the files > still > >>> > get written again. > >>> > > >>> > Here is the route I am using: > >>> > > >>> > public void ftpToLocal() throws Exception { > >>> > runRoutes(new RouteBuilder() { > >>> > @Override > >>> > public void configure() throws Exception { > >>> > Registry registry = getContext().getRegistry(); > >>> > String user = (String) registry.lookup("ftpUser"); > >>> > String pass = (String) registry.lookup("ftpPass"); > >>> > > >>> > // we use a delay of 60 minutes (eg. once pr. hour we poll the > >>> FTP > >>> > server > >>> > // long delay = 60 * 60 * 1000L; > >>> > // long delay = TimeUnit.SECONDS.toMillis(1); > >>> > long delay = 200; > >>> > > >>> > from("sftp://"+user+"@example.com/path/to/dir?password="; + > pass > >>> + > >>> > "&binary=true&delay="+delay+"&consumer.delay=" + delay) > >>> > .to("file://data/ftp?fileExist=Ignore"); > >>> > } > >>> > }); > >>> > } > >>> > > >>> > > >>> > Thanks for your time reading this email. > >>> > >>> > >>> > >>> -- > >>> Claus Ibsen > >>> ----------------- > >>> FuseSource > >>> Email: cib...@fusesource.com > >>> Web: http://fusesource.com > >>> Twitter: davsclaus, fusenews > >>> Blog: http://davsclaus.com > >>> Author of Camel in Action: http://www.manning.com/ibsen > >>> > >> > > > > -- > Claus Ibsen > ----------------- > FuseSource > Email: cib...@fusesource.com > Web: http://fusesource.com > Twitter: davsclaus, fusenews > Blog: http://davsclaus.com > Author of Camel in Action: http://www.manning.com/ibsen >
