Hi,
I think you can do this using a idempotent repository:
http://camel.apache.org/idempotent-consumer.html
from("ftp:foo")
.split(body().tokenize("\n")
.idempotentConsumer(header("myMessageId"),MemoryIdempotentRepository.memoryIdempotentRepository(200))
.to("jms:bar");
in your case the memory idempotent consumer is not really suitable, but
you can use a file based one, or a db based (we've writen one for mongo
db: https://github.com/catify/camel-mongodb ;). So you don't have to
rollback, because the idempotent consumer will avoid dublicated line
consumption, even if you start consuming a file a second or third time.
Best regards - Claus
On 26.11.2010 14:22, Andreas A. wrote:
Hi
I have an application that simply put does this:
Ingoing:
1 - Fetch file from FTP->Local directory.
2 - Read file from Local directory, transform and split the file into
several messages and deliver to JMS queue.
Outgoing:
1 - Fetch messages from JMS queue as they are produced, transform and store
in local directory one by one.
2 - Twice a day files are read from local directory, compiled into a list
(one file) of messages and delivered to FTP.
The problem is that when something goes wrong it is a nightmare to figure
out what has been sent and recieved and what has not. The key issues are: We
do not want duplicates input to JMS queue. We do not want duplicates output
to FTP. And also traceability of the messages.
There are several spots where this can go wrong at the moment.
1 - If JMS -> File fails, message is lost. Can I use the transaction manager
here?
2 - If split fails from file -> JMS the messages already put to the JMS
queue will stay, and file will rollback. If we try and run the file again,
there will be duplicate messages on the queue. Can I use the transaction
manager here?
3 - If the Aggregator fails, the route that called it does not rollback, all
messages are lost, why aren't they being rolled back?
Basically what I want is for either JMS messages or files to always roll
completely back to the "queue" from where they were taken if something goes
wrong. That way I can control state of the process based on their position
in either a directory or a JMS queue. At the moment if there's an error I
need to go through logs and see what has ended up where, and retrieve
payloads from the logfiles.
Are these routes design dead, or can I fix this somehow.
