How to read data from a paginated API and upload it as multipart file to S3 page by page

2023-07-31 Thread Sahil Upneja
Hi Everyone ! I am trying to read data from a paginated API , Since the whole data is huge so I want to read one page from paginated API and upload it as multipart file to S3 and then read 2nd page & upload it to S3 and so on. Could you please let me know if this usecase is achievable through Cam

Connection Pooling in (S)FTP Component

2023-07-31 Thread ICS460-BAI
Hello, due to the fact that we have many transports on a single server we need the ability to pool/reuse ftp connections. Otherwise, we have to many reconnects or to many open connections! Do you have any recommendations how we can realize such a connection pool. We are currently trying to pool

Fwd: loading routes from external xml files

2023-07-31 Thread Ricardo Garcia
Hi, I am migrating to camel 4, as part of a broader migration to all jakarta API instead of javax API In camel 3 I could load routes from multiple external xml files: try (DirectoryStream ds = Files.newDirectoryStream(routesDir, "*.xml" ); DefaultResourceResolvers.FileResolver resolver = new Defau

Re: loading routes from external xml files

2023-07-31 Thread Claus Ibsen
Hi You can use org.apache.camel.support.PluginHelper On Mon, Jul 31, 2023 at 11:04 PM Ricardo Garcia wrote: > Hi, I am migrating to camel 4, as part of a broader migration to all > jakarta API instead of javax API > In camel 3 I could load routes from multiple external xml files: > > try (Direc

Re: How to read data from a paginated API and upload it as multipart file to S3 page by page

2023-07-31 Thread Claussnitzer, Ralf
Hi Sahil, as far as I know the HTTP producer does not support polling based on headers. You could trigger the individual HTTP requests by using sort of an event loop: Event/Message driven consumer route (direct or jms) that makes the actual request and emits a follow-up event for the next page ba

Re: loading routes from external xml files

2023-07-31 Thread ski n
OK, good to know as I use this functionality also heavily. Some questions/remarks about this: 1. What's the reason that it was (re)moved? Is there a new concept behind the PluginHelper? 2. Maybe good to add this to the migration guide ( https://camel.apache.org/manual/camel-4-migration-guide.html)

EventNotifier and direct/seda endpoints in Camel 4

2023-07-31 Thread ski n
Currently, I use the EventNotifier to collect events in Camel: https://www.javadoc.io/doc/org.apache.camel/camel-api/3.21.0/org/apache/camel/spi/CamelEvent.html I have integrations that consist of 1 or more routes like this: To route between two routes I use either: - direct-vm -

Re: EventNotifier and direct/seda endpoints in Camel 4

2023-07-31 Thread Claus Ibsen
Hi No there is not On Tue, Aug 1, 2023 at 8:49 AM ski n wrote: > Currently, I use the EventNotifier to collect events in Camel: > > > https://www.javadoc.io/doc/org.apache.camel/camel-api/3.21.0/org/apache/camel/spi/CamelEvent.html > > I have integrations that consist of 1 or more routes like t