Re: Flatpack and aggregation

2013-08-07 Thread Tom.Fornoville 
Hi Zemian,

Thanks for putting me on the right track with the idempotentConsumer!

Since Flatpack puts the records in a Java Map in the Body I had to do a
convertTo and ended up with the following:

bean id=organisationMemoryRepository
class=org.apache.camel.processor.idempotent.MemoryIdempotentRepository /
...
route id=process-line
from uri=flatpack:fixed:kbo.pzmap.xml /
convertBodyTo type=java.util.Map/
filter
ognlrequest.body[ACTION] == 070/ognl
idempotentConsumer messageIdRepositoryRef=organisationMemoryRepository
ognlrequest.body[ORGANISATION]/ognl
to uri=log:be.roots.organisation?level=INFO /
!-- TODO: send it to the webservice --
/idempotentConsumer
/filter
/route

This seems to work as expected :-)

Cheers,
Tom

Tom Fornoville
Senior Developer
m: +32 478 65 86 51
www.roots.be


On Wed, Aug 7, 2013 at 3:46 AM, saltnlight5 [via Camel] 
ml-node+s465427n5736874...@n5.nabble.com wrote:

 Hello Tom,

 I think you can try flatpack with one of the idempotent consumer (
 http://camel.apache.org/idempotent-consumer.html) to eliminate duplicated
 messages. For example, maybe something like this:

   from(file:inbox)
 .to(flatpack:fixed:yourdata.pzmap.xml)
 .filter(header(action_id))
 .idempotentConsumer(header(organisation_id),
 MemoryIdempotentRepository.memoryIdempotentRepository(200))

 Also alternative to flatpack, you may also checkout Camel bindy:
 http://camel.apache.org/bindy.html to parse fixed length records.

 Have fun,
 Zemian

 --
  If you reply to this email, your message will be added to the discussion
 below:

 http://camel.465427.n5.nabble.com/Flatpack-and-aggregation-tp5736837p5736874.html
  To unsubscribe from Flatpack and aggregation, click 
 herehttp://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=5736837code=dG9tLmZvcm5vdmlsbGVAcm9vdHMuYmV8NTczNjgzN3wxNzA0MjEyMTky
 .
 NAMLhttp://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml





--
View this message in context: 
http://camel.465427.n5.nabble.com/Flatpack-and-aggregation-tp5736837p5736914.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Flatpack and aggregation

2013-08-06 Thread Tom.Fornoville 
'm relatively new to Camel and guessing my problem can be solved with a
combination of FlatPack and an aggregation but I'm not seeing how I can make
it work without writing a lot of code.

It starts with a fixed width file with 2 columns (actually more but only 2
are relevant): action_id and organisation_id. The file contains a couple
thousand lines and can contain duplicates.

2 things needed:

filter out lines for a specific action_id (only action_id 70 is of interest
for us)
deduplicate based on organisation_id (the same organisation can have appear
on multiple lines)
It will be greatly appreciated if anyone has a solution or can point me in
the right direction .

Cheers, Tom



--
View this message in context: 
http://camel.465427.n5.nabble.com/Flatpack-and-aggregation-tp5736837.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Flatpack and aggregation

2013-08-06 Thread saltnlight5 
Hello Tom,

I think you can try flatpack with one of the idempotent consumer
(http://camel.apache.org/idempotent-consumer.html) to eliminate duplicated
messages. For example, maybe something like this:

  from(file:inbox)
.to(flatpack:fixed:yourdata.pzmap.xml)
.filter(header(action_id))
.idempotentConsumer(header(organisation_id),
MemoryIdempotentRepository.memoryIdempotentRepository(200))

Also alternative to flatpack, you may also checkout Camel bindy:
http://camel.apache.org/bindy.html to parse fixed length records.

Have fun,
Zemian



--
View this message in context: 
http://camel.465427.n5.nabble.com/Flatpack-and-aggregation-tp5736837p5736874.html
Sent from the Camel - Users mailing list archive at Nabble.com.