Re: Integrating Apache Beam and Apache Camel

2022-11-16 Thread Claudio Miranda
On Wed, Nov 16, 2022 at 1:57 PM John Casey wrote: > > from(something1?).to(some:existing:camel:component) > from(some:existing:camel:component?with=config).to(something2?) Look at the "To Dynamic" endpoint, you can set the endpoint string at runtime. https://camel.apache.org/components/3.18.x/ei

Re: Integrating Apache Beam and Apache Camel

2022-11-16 Thread ski n
I agree with Romain that processor is good thing to look at. Here are some resources that maybe useful: https://stackoverflow.com/questions/20578756/apache-camel-processor-purpose https://camel.apache.org/manual/processor.html https://www.youtube.com/watch?v=s_6ip_9Yz5M https://access.redhat.com/

Re: Integrating Apache Beam and Apache Camel

2022-11-16 Thread Romain Manni-Bucau
Hi John, Last time I implemented this bridge i moved from the templates to directly using the processor (advantage being able to use sync or async send depending how you integrate it with beam - in multicast mode if you have only async processors you can call them all and await all to finish witho

Re: Integrating Apache Beam and Apache Camel

2022-11-16 Thread John Casey
And an hour after I send this, I stumble upon ProducerTemplate and ConsumerTemplate, which I'm pretty sure are exactly what I'm looking for. On Wed, Nov 16, 2022 at 11:56 AM John Casey wrote: > Hi Camel, > > I'm John, working on the Apache Beam project. I'm looking to write a Beam > wrapper for

Integrating Apache Beam and Apache Camel

2022-11-16 Thread John Casey
Hi Camel, I'm John, working on the Apache Beam project. I'm looking to write a Beam wrapper for some Camel components, as Camel has a very well developed ecosystem of connectors that we are looking to borrow. I essentially want to inject messages into camel, or extract them, directly. Examples w

RE: How can i retrieve file name when SftpException Occured

2022-11-16 Thread Daniel Langevin
Hi, Usually the current message should contain the CamelFileName header when catching exception. If not, before the SFTP transfer you can keep the filename in a property and reuse it afterwards. TEST {{fileprefixABC}}_${date:now:MMdd_HHmmss}.txt ftp://{{serverABC}}:{{portNumberABC}}{{di

RE: Re: Re: Offset in OffsetRepositoy doesn't get increase after manual commit

2022-11-16 Thread Yuttana Sangchoei
Hi Otavio, Thank you for creating the ticket. I found where the problem might be in 3.18.3. At CommitManager in createCommitManager static method, there are some check starting by checking if isAllowManualCommit, if so it continues checking the type of manualCommitFactory. However, if not, it c