Here is a small junit test which explains the problem:
public void testIdempotentConsumerDoesntIgnoreRecursiveDuplicates() throws
Exception {
String newFile1 = "some_dir/some_file3";
String newFile2 = "some_file3";
mockRecognizerService.setExpectedMessageCount(2);
dropFileInTargetDir(newFile1);
dropFileInTargetDir(newFile2);
Thread.sleep(4000);
mockRecognizerService.assertIsSatisfied();
}
dropFileInTargetDir - creates a file under the directory which the file
component is listening to.
This test fails.
sagy wrote:
>
> exactly. Lets assume a directory root called /test
> the writing application will write the following files:
> /test/file1.txt
> /test/file2.txt
> /test/sub-folder/file1.txt
>
> etc..
> My application will get an event for
> /test/file1.txt
> /test/file2.txt
>
> but not for
> /test/sub-folder/file1.txt
>
> since the IdempotentRepository already contains an entry called
> "file1.txt". It was created for the /test/file1.txt event.
>
>
>
>
>
> Claus Ibsen-2 wrote:
>>
>> On Tue, Apr 13, 2010 at 5:10 PM, sagy <[email protected]> wrote:
>>>
>>> I'm trying to consume files from it not skip it.
>>> This is a general description of the situation:
>>>
>>> There is another application with write permissions which create files
>>> in
>>> that directory and sub directories.
>>> My application only has read permissions but I still need to consume the
>>> files from that directory structure.
>>>
>>> That's why I'm using the "noop=true" flag. I can't move the files or
>>> delete
>>> them.
>>> It seems from the documentation that noop is my only choice.
>>> But a combination of "recursive=true" and "noop=true" causes the problem
>>> I've described in the original post.
>>>
>>
>> And that app can write files with the same name?
>>
>>
>>
>>>
>>>
>>> Claus Ibsen-2 wrote:
>>>>
>>>> On Tue, Apr 13, 2010 at 3:33 PM, sagy <[email protected]> wrote:
>>>>>
>>>>> I'm actually trying to consume files from a directory recursively and
>>>>> the
>>>>> directory has read-only permissions.
>>>>> Is there another way to do it without using noop=true&recursive=true ?
>>>>>
>>>>
>>>> You can filter out the read-only directory if you should "skip" it.
>>>>
>>>>
>>>>> Thanks,
>>>>> Sagy
>>>>>
>>>>>
>>>>>
>>>>> Claus Ibsen-2 wrote:
>>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Yeah its on purpose for some reason, which as I can't really recall
>>>>>> right
>>>>>> now.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Apr 13, 2010 at 2:33 PM, sagy <[email protected]> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>> I'm using Camel 2.2 and have the following route:
>>>>>>>
>>>>>>> from("file://test?recursive=true&noop=true").process(...
>>>>>>>
>>>>>>> When I drop a file f1.txt into the test folder the processor gets
>>>>>>> called.
>>>>>>> When I drop the file f2.txt into a sub folder of test folder for
>>>>>>> example
>>>>>>> test/sub/f2.txt ,again the processor gets called.
>>>>>>> But when I drop the file f1.txt into test/sub the processor doesn't
>>>>>>> get
>>>>>>> called.
>>>>>>> This is happening because of line 115 in camel-core
>>>>>>> org.apache.camel.component.file.GenericFileOnCompletion:
>>>>>>>
>>>>>>> // only add to idempotent repository if we could process
>>>>>>> the
>>>>>>> file
>>>>>>> // only use the filename as the key as the file could be
>>>>>>> moved
>>>>>>> into a done folder
>>>>>>>
>>>>>>> endpoint.getIdempotentRepository().add(file.getFileName());
>>>>>>>
>>>>>>> The repository stores just the file name without the path.
>>>>>>> Therefore if 2 files with the same name are dropped to 2 different
>>>>>>> sub
>>>>>>> directories only the first one will be processed.
>>>>>>> Is this the intended behavior?
>>>>>>> Is there a way to work around it?
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> View this message in context:
>>>>>>> http://old.nabble.com/File-consumer-with-noop%3Dtrue-recursive%3Dtrue-tp28229501p28229501.html
>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://old.nabble.com/File-consumer-with-noop%3Dtrue-recursive%3Dtrue-tp28229501p28230077.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/File-consumer-with-noop%3Dtrue-recursive%3Dtrue-tp28229501p28231610.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
>
--
View this message in context:
http://old.nabble.com/File-consumer-with-noop%3Dtrue-recursive%3Dtrue-tp28229501p28232225.html
Sent from the Camel - Users mailing list archive at Nabble.com.
