Hi
For Camel in general there is docs on the website, books, videos on you-tube and so forth. This is a volunteer based community, and a bunch of core engineers that are very busy. I am afraid we don't have excess time to help when its too general and environment specific questions/ There are commercial companies you can contract to get assistance https://camel.apache.org/manual/commercial-camel-offerings.html On Fri, Jan 31, 2025 at 2:49 PM Vineet Kumar <vineet.ku...@iongroup.com.invalid> wrote: > Hello Team, > > I am reaching out on behalf of ION Group regarding some issues we've > encountered while working with Apache Camel in our Kafka-based projects. We > are seeking your assistance to resolve some challenges related to exception > handling in both producer and consumer routes. We have been working with a > Kafka producer and consumer route in Camel, and our goal is to catch > exceptions if something goes wrong in the Kafka producer or consumer routes. > > > *Project 1: *Latest Version of Libraries (Camel: 4.8.0, Kafka Clients: > 3.9.0) - Template POC Project > *Findings:* > > - *Producer Route:* We successfully implemented exception handling for > the producer route by adding onException block on global level with > maxBlockMs=2500 config. > - *This allowed us to catch exceptions within 2.5 seconds if any error > occurred during polling, which triggered the OnException block.* > - *Consumer Route:* Unfortunately, we were unable to achieve > similar results with the consumer route. Even when reproducing an exception > (e.g., *UnknownTopicOrPartitionException*) > - *The consumer route continued polling indefinitely, and the > OnException block was never triggered.* > - While we found that we could capture exceptions like * > UnknownTopicOrPartitionException *for the producer route, the same did > not apply to the consumer route. > - There doesn't seem to be a similar config (like maxBlockMs) for the > consumer route, and we're unable to handle exceptions in the same way. > - We were only able to catch > *org.apache.kafka.common.errors.TimeoutException > **on which we invoked getCause method to get root cause wrapped > exception** i.e, UnknownTopicOrPartitionException* > > > > *Project 2: *Older Version of Libraries (Camel: 3.14.10, Kafka Clients: > 2.7.2) - *Actual version we want to use* > *Findings:* > > - *Producer Route:* We were able to reproduce a timeout exception, but > when calling .getCause() on the Throwable object, we received a null > result. > - In this version, only the TimeoutException was captured, and the > root cause was not available via getCause method invocation. > - *Consumer Route:* Similarly, we were unable to catch any > exceptions in the consumer route, which resulted in indefinite polling. > - We used similar example like producer: We tried consuming from a > topic which doesn't exist on the kafka broker > - Note:* auto.create.topics.enable=false* was added explicitly in > server.properties file used by kafka broker to prevent auto > creation of > topic > > > > *Steps We've Taken:* To resolve these issues, we've tried the following > below approaches in both versions and none of them seems to work out for us > except onException for producer just for TimeoutException. > > 1. *OnException: Only caught timeout exceptions in the producer route > (Only timeout exceptions)* > 2. *ErrorHandler: We tested both custom error and exception handlers > without success.* > 3. *PollExceptionStrategy* > 4. *BreakOnFirstError* > 5. *PollOnError* > 6. *BridgeErrorHandler* > 7. *ExceptionHandler* > > > > *Questions:* We would appreciate your guidance on the following: > > 1. *Are we missing any key configurations or settings that could help > resolve this issue to catch any type of exception defined in Errors.class > file under kafka-clients library?* > 1. *Includes both retriable and non-retriable exceptions* > 2. *Could there be an error in our setup or configuration?* > 3. *Is there any relevant information missing from the Camel > documentation that could help us handle these exceptions properly?* > 4. *Is this a known issue or a potential bug in the Camel version > we're using in both projects?* > 5. *Any other suggestions or insights you may have would be greatly > appreciated.* > > > For your reference, I've attached the project zip file on the below Zulip > chat link containing relevant information, screenshot including the > dependency version details and all necessary project files. > Zulip chat link: (46) #general > Assistance Required on Exception > Handling in Camel Kafka Con - camel - Zulip > <https://camel.zulipchat.com/#narrow/channel/257295-general/topic/Assistance.20Required.20on.20Exception.20Handling.20in.20Camel.20Kafka.20Con> > Thank you in advance for your time and support. We look forward to your > guidance on this matter. > > > Regards, > [image: ION] > *Vineet Kumar* > vineet.ku...@iongroup.com > *m* > +91 9003801369 > iongroup.com <https://www.iongroup.com/> > > [image: Impact] > <https://www.linkedin.com/showcase/ioncommodities/posts/?feedView=all> > > -- Claus Ibsen ----------------- @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
