Unfortunately we cannot open more than one connection at time, so it is not possible to have two consumers reading the two (actually now it is three) files at the same time.
IMHO this sounds a bit odd/dangerous way of transferring data between > parties. > There is no good way to tell if these 2 files are in sync, or the > first file has new content, and the 2nd has previous content. > > It would be better if the other party could change its strategy to > upload new files using a name pattern to pair the files, eg > a sequence number that would match etc. > > myfile-123.dat > myotherfile-123.dat > > Or to use a 3rd "done" file to tell you that the 2 files has now been > properly written. eg deleting the done file first, write the 2 files, > and save the done file again. > Sequencing would be great, but we will not have it. > > > Can you move the files before you download them? As a move operation > on a remote FTP server is faster than having to download the 2 files > in parallel in hopes that you make it before the other party starts to > override the files. > > If you can move the files (or rename) then thats likely a near atomic > IO operation, and thus would be fast doing 2 file moves. Although you > would need to send 2 FTP commands. > No, we cannot move or rename the files, because they are consumed also by other parties, we don't know about. > > Camel has a preMove option, but its still based on the > one-file-at-a-time strategy the ftp consumer uses. > > > If you know the file names in advance, then you can have 2 routes, > each route picking up the designated file. And if you are allowed to > delete the file after download, then you could assume if there is a > new file then its new content since last. > > But the 2 sec frequency is a bit fast, which mean you would need to > download and delete the file < 2s. And have the polling frequency of > the ftp consumer aligned with the other party. > Can you tell me what is 2 sec frequency, where? What would be ideal is, if we able to lock the files we want to read before we start downloading them. For example open the two files, and then start the download. The only guarantee we have is that: a file will not be overridden while we are reading it. Do you thing it is possible to extend the component, so that it opens more than one file, and then start reading them? Bilgin > > > > > Thanks > > Bilgin > > > > -- > Claus Ibsen > ----------------- > FuseSource > Email: cib...@fusesource.com > Web: http://fusesource.com > Twitter: davsclaus, fusenews > Blog: http://davsclaus.com > Author of Camel in Action: http://www.manning.com/ibsen >
