Re: Camel Java DSL Cheat
All examples can be used for Camel documentation as well (same license as Camel). Think it should be reviewed first though. Regards, Raymond (Ski n) Op wo 13 jan. 2021 om 07:05 schreef Ajmera, Hemang C : > Hi >This seems to be really useful. I think we should include this in camel > documentation, so that it can be easily reference by anyone who is new to > camel. > > > Thanks and Regards, > Hemang Ajmera > > > -Original Message- > From: ski n > Sent: 11 January 2021 19:45 > To: users@camel.apache.org > Subject: Camel Java DSL Cheat > > > EXTERNAL SENDER: Do not click any links or open any attachments unless > you trust the sender and know the content is safe. > EXPÉDITEUR EXTERNE:Ne cliquez sur aucun lien et n’ouvrez aucune pièce > jointe à moins qu’ils ne proviennent d’un expéditeur fiable, ou que vous > ayez l'assurance que le contenu provient d'une source sûre. > > Hi All, > > For a project I created a lot of Camel routes. Sometimes I noticed that I > need to look up how to do something. To look it up more quickly I created a > cheat sheet for the Java DSL. > > Maybe also useful for others: > > > https://urldefense.com/v3/__https://github.com/assimbly/camel-examples__;!!AaIhyw!4nwellmXk5yNYCygr4_WfufWZQxWS7zJPDitTEVxM6OD72BqL1LDAkz30H74Dker$ > > In case you miss something just make a PR. > > Kind regards, > > Raymond >
Re: Camel-K / Kafka / Avro: something broke overnight
Hi Mark,
indeed it looks like the additionalProperties option is broken in 3.7.0. I
am pushing a fix for both branches, 3.7.x and master branch.
Regards,
Omar
On Thu, Jan 14, 2021 at 6:41 PM Andrea Cosentino wrote:
> I don't think it is feasible from our side to get to 3.6.0, it's up to you.
>
> Can you try to set the additionalProperties at component level like this
> example?
>
>
> https://github.com/apache/camel/blob/master/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/KafkaComponentTest.java#L63
>
> Il giorno gio 14 gen 2021 alle ore 17:28 mark ha
> scritto:
>
> > Hello,
> > Until yesterday we were using Camel-K to send Avro-encoded messages to
> > Kafka using code of the form,
> >
> > .to("kafka:{{topic}}?brokers={{kafka-bootstrap}}"
> > +
> >
> >
> "&keySerializerClass=org.apache.kafka.common.serialization.StringSerializer"
> > +
> > "&serializerClass=io.apicurio.registry.utils.serde.AvroKafkaSerializer"
> > + "&additionalProperties.apicurio.registry.url=" +
> > schemaRegistry
> >
> > Running this today, our Integration had picked up camel-kafka 3.7.0 and
> > issue https://issues.apache.org/jira/browse/CAMEL-15770 which changed
> the
> > first two properties to keySerializer and valueSerializer. However
> > adjusting the code to,
> >
> > .to("kafka:{{topic}}?brokers={{kafka-bootstrap}}"
> > +
> > "&keySerializer=org.apache.kafka.common.serialization.StringSerializer"
> > +
> > "&valueSerializer=io.apicurio.registry.utils.serde.AvroKafkaSerializer"
> > + "&additionalProperties.apicurio.registry.url=" + schemaRegistry
> >
> > Leaves us no longer able to set additionalProperties. Attempts such as
> > adding
> >
> > -d mvn:org.apache.camel/camel-kafka:[3.6.0]
> >
> > have no effect: we always get camel-kafka 3.7 and cannot pin the
> > Integration to 3.6 or below. The relevant stack trace is below - please
> can
> > anyone help us either adjust the apicurio.registry.url for the new
> library,
> > or pin things back to 3.6 please?
> >
> > [1] Caused by: org.apache.kafka.common.KafkaException: Failed to
> construct
> > kafka producer
> > [1] at
> >
> >
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:434)
> > [1] at
> >
> >
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:298)
> > [1] at
> >
> >
> org.apache.camel.component.kafka.KafkaProducer.doStart(KafkaProducer.java:115)
> > [1] at
> > org.apache.camel.support.service.BaseService.start(BaseService.java:115)
> > [1] at
> >
> >
> org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:84)
> > [1] at
> >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.internalAddService(AbstractCamelContext.java:1425)
> > [1] at
> >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.addService(AbstractCamelContext.java:1343)
> > [1] at
> > org.apache.camel.processor.SendProcessor.doStart(SendProcessor.java:236)
> > [1] at
> > org.apache.camel.support.service.BaseService.start(BaseService.java:115)
> > [1] at
> >
> >
> org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:84)
> > [1] at
> >
> >
> org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:101)
> > [1] at
> >
> >
> org.apache.camel.processor.errorhandler.RedeliveryErrorHandler.doStart(RedeliveryErrorHandler.java:1487)
> > [1] at
> >
> >
> org.apache.camel.support.ChildServiceSupport.start(ChildServiceSupport.java:60)
> > [1] ... 37 more
> > [1] Caused by: java.lang.IllegalArgumentException: Missing registry base
> > url, set apicurio.registry.url
> > [1] at
> >
> >
> io.apicurio.registry.utils.serde.AbstractKafkaSerDe.configure(AbstractKafkaSerDe.java:120)
> > [1] at
> >
> >
> io.apicurio.registry.utils.serde.AbstractKafkaStrategyAwareSerDe.configure(AbstractKafkaStrategyAwareSerDe.java:75)
> > [1] at
> >
> >
> io.apicurio.registry.utils.serde.AvroKafkaSerializer.configure(AvroKafkaSerializer.java:78)
> > [1] at
> >
> >
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:369)
> > [1] ... 49 more
> >
> > My dependencies are currently,
> >
> > // camel-k:
> > dependency=mvn:org.apache.camel.quarkus/camel-quarkus-kafka:1.5.0
> > // camel-k: dependency=mvn:org.apache.avro/avro:1.10.1
> > // camel-k: dependency=mvn:org.glassfish.jersey.core/jersey-common:2.22.2
> > // camel-k:
> > dependency=mvn:io.apicurio/apicurio-registry-utils-serde:1.3.2.Final
> >
> > Many thanks in advance,
> > Regards,
> >
> > Mark
> >
>
Re: Camel-K / Kafka / Avro: something broke overnight
Andrea,
Thank you very much for the pointer. Yes we've hacked something
together using the test code you linked above, and a hand-rolled
deserializer based on
https://github.com/Apicurio/apicurio-registry/blob/1.3.x/app/src/test/java/io/apicurio/registry/RegistrySerdeTest.java
.
Omar,
Thank you for confirming the regression, and for preparing a fix.
Regards,
Mark
On Fri, 15 Jan 2021 at 09:23, Omar Al-Safi wrote:
> Hi Mark,
>
> indeed it looks like the additionalProperties option is broken in 3.7.0. I
> am pushing a fix for both branches, 3.7.x and master branch.
>
> Regards,
> Omar
>
>
> On Thu, Jan 14, 2021 at 6:41 PM Andrea Cosentino
> wrote:
>
> > I don't think it is feasible from our side to get to 3.6.0, it's up to
> you.
> >
> > Can you try to set the additionalProperties at component level like this
> > example?
> >
> >
> >
> https://github.com/apache/camel/blob/master/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/KafkaComponentTest.java#L63
> >
> > Il giorno gio 14 gen 2021 alle ore 17:28 mark ha
> > scritto:
> >
> > > Hello,
> > > Until yesterday we were using Camel-K to send Avro-encoded messages to
> > > Kafka using code of the form,
> > >
> > > .to("kafka:{{topic}}?brokers={{kafka-bootstrap}}"
> > > +
> > >
> > >
> >
> "&keySerializerClass=org.apache.kafka.common.serialization.StringSerializer"
> > > +
> > > "&serializerClass=io.apicurio.registry.utils.serde.AvroKafkaSerializer"
> > > + "&additionalProperties.apicurio.registry.url=" +
> > > schemaRegistry
> > >
> > > Running this today, our Integration had picked up camel-kafka 3.7.0 and
> > > issue https://issues.apache.org/jira/browse/CAMEL-15770 which changed
> > the
> > > first two properties to keySerializer and valueSerializer. However
> > > adjusting the code to,
> > >
> > > .to("kafka:{{topic}}?brokers={{kafka-bootstrap}}"
> > > +
> > > "&keySerializer=org.apache.kafka.common.serialization.StringSerializer"
> > > +
> > > "&valueSerializer=io.apicurio.registry.utils.serde.AvroKafkaSerializer"
> > > + "&additionalProperties.apicurio.registry.url=" +
> schemaRegistry
> > >
> > > Leaves us no longer able to set additionalProperties. Attempts such as
> > > adding
> > >
> > > -d mvn:org.apache.camel/camel-kafka:[3.6.0]
> > >
> > > have no effect: we always get camel-kafka 3.7 and cannot pin the
> > > Integration to 3.6 or below. The relevant stack trace is below - please
> > can
> > > anyone help us either adjust the apicurio.registry.url for the new
> > library,
> > > or pin things back to 3.6 please?
> > >
> > > [1] Caused by: org.apache.kafka.common.KafkaException: Failed to
> > construct
> > > kafka producer
> > > [1] at
> > >
> > >
> >
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:434)
> > > [1] at
> > >
> > >
> >
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:298)
> > > [1] at
> > >
> > >
> >
> org.apache.camel.component.kafka.KafkaProducer.doStart(KafkaProducer.java:115)
> > > [1] at
> > >
> org.apache.camel.support.service.BaseService.start(BaseService.java:115)
> > > [1] at
> > >
> > >
> >
> org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:84)
> > > [1] at
> > >
> > >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.internalAddService(AbstractCamelContext.java:1425)
> > > [1] at
> > >
> > >
> >
> org.apache.camel.impl.engine.AbstractCamelContext.addService(AbstractCamelContext.java:1343)
> > > [1] at
> > >
> org.apache.camel.processor.SendProcessor.doStart(SendProcessor.java:236)
> > > [1] at
> > >
> org.apache.camel.support.service.BaseService.start(BaseService.java:115)
> > > [1] at
> > >
> > >
> >
> org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:84)
> > > [1] at
> > >
> > >
> >
> org.apache.camel.support.service.ServiceHelper.startService(ServiceHelper.java:101)
> > > [1] at
> > >
> > >
> >
> org.apache.camel.processor.errorhandler.RedeliveryErrorHandler.doStart(RedeliveryErrorHandler.java:1487)
> > > [1] at
> > >
> > >
> >
> org.apache.camel.support.ChildServiceSupport.start(ChildServiceSupport.java:60)
> > > [1] ... 37 more
> > > [1] Caused by: java.lang.IllegalArgumentException: Missing registry
> base
> > > url, set apicurio.registry.url
> > > [1] at
> > >
> > >
> >
> io.apicurio.registry.utils.serde.AbstractKafkaSerDe.configure(AbstractKafkaSerDe.java:120)
> > > [1] at
> > >
> > >
> >
> io.apicurio.registry.utils.serde.AbstractKafkaStrategyAwareSerDe.configure(AbstractKafkaStrategyAwareSerDe.java:75)
> > > [1] at
> > >
> > >
> >
> io.apicurio.registry.utils.serde.AvroKafkaSerializer.configure(AvroKafkaSerializer.java:78)
> > > [1] at
> > >
> > >
> >
> org.apache.kafka.clients.producer.KafkaProducer.(KafkaProducer.java:369)
> > > [1] ... 49 more
> > >
> > > My dependencies are currently,
> > >
> > > // camel-k:
> > > dependency=mvn:org.apache.camel.quarkus/camel-quarkus-kafka:1.5.0
> > > // camel-k: depend
Re: SupervisingRouterController not working as expectation (ApacheCamel:3.7.0)
@Claus Ibsen Thanks for clarification, For timer I agree as it already started so SupervisingRouterController will not take care, but for kubernetes route is failing at connection level but still SupervisingRouterController is not retrying, But SupervisingRouterController retry work for my third part of broker route 2021-01-15 17:55:18,333 ERROR [org.apa.cam.pro.err.DefaultErrorHandler] (Camel (camel-1) thread #1 - timer://ipc) Failed delivery for (MessageId: C430425BC77567B- on ExchangeId: C430425BC77567B-). Exhausted after delivery attempt: 1 caught: io.fabric8.kubernetes.client.KubernetesClientException: Operation: [get] for kind: [Secret] with name: [broker] in namespace: [production] failed. Message History (complete message history is disabled) --- RouteId ProcessorId Processor Elapsed (ms) [main-route] [main-route] [from[timer://ipc?delay=6&repeatCount=1] ] [ 4813] ... [GetSecretsAndYQCon] [to5 ] [kubernetes-secrets:///?kubernetesClient=#kubernetesClient&operation=getSecret ] [ 0] Stacktrace --- : io.fabric8.kubernetes.client.KubernetesClientException: Operation: [get] for kind: [Secret] with name: [broker] in namespace: [production] failed. at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:225) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:186) at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:84) at org.apache.camel.component.kubernetes.secrets.KubernetesSecretsProducer.doGetSecret(KubernetesSecretsProducer.java:130) at org.apache.camel.component.kubernetes.secrets.KubernetesSecretsProducer.process(KubernetesSecretsProducer.java:72) at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66) at org.apache.camel.processor.SendProcessor.lambda$process$2(SendProcessor.java:188) at org.apache.camel.support.cache.DefaultProducerCache.doInAsyncProducer(DefaultProducerCache.java:317) at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:187) at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:395) at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148) at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60) at org.apache.camel.processor.Pipeline.process(Pipeline.java:147) at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:312) at org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:207) at org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76) at java.util.TimerThread.mainLoop(Timer.java:555) at java.util.TimerThread.run(Timer.java:505) Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:994) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:336) at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:300) at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:185) at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:224) at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.java:108) at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.java:88) at okhttp3.internal.connection.Transmitter.newExchange(Transmitter.java:169) at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:41) at okhttp3.internal.h
Re: SupervisingRouterController not working as expectation (ApacheCamel:3.7.0)
Hi If you want to retry the kubernetes endpoint, then you can use For example to try 10 times errorHandler().maximumRedeliveries(10); See more at https://camel.apache.org/manual/latest/defaulterrorhandler.html https://camel.apache.org/components/latest/eips/dead-letter-channel.html On Fri, Jan 15, 2021 at 7:20 PM Imran Raza Khan wrote: > > @Claus Ibsen > > Thanks for clarification, For timer I agree as it already started so > SupervisingRouterController will not take care, but for kubernetes route is > failing at connection level but still SupervisingRouterController is not > retrying, But SupervisingRouterController retry work for my third part of > broker route > > > 2021-01-15 17:55:18,333 ERROR [org.apa.cam.pro.err.DefaultErrorHandler] > (Camel (camel-1) thread #1 - timer://ipc) Failed delivery for (MessageId: > C430425BC77567B- on ExchangeId: > C430425BC77567B-). Exhausted after delivery attempt: 1 > caught: io.fabric8.kubernetes.client.KubernetesClientException: Operation: > [get] for kind: [Secret] with name: [broker] in namespace: [production] > failed. > > Message History (complete message history is disabled) > > --- > RouteId ProcessorId Processor >Elapsed (ms) > [main-route] [main-route] > [from[timer://ipc?delay=6&repeatCount=1] > ] [ 4813] > ... > [GetSecretsAndYQCon] [to5 ] > [kubernetes-secrets:///?kubernetesClient=#kubernetesClient&operation=getSecret > ] [ 0] > > Stacktrace > > --- > : io.fabric8.kubernetes.client.KubernetesClientException: Operation: > [get] for kind: [Secret] with name: [broker] in namespace: [production] > failed. > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:72) > at > io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:225) > at > io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:186) > at > io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:84) > at > org.apache.camel.component.kubernetes.secrets.KubernetesSecretsProducer.doGetSecret(KubernetesSecretsProducer.java:130) > at > org.apache.camel.component.kubernetes.secrets.KubernetesSecretsProducer.process(KubernetesSecretsProducer.java:72) > at > org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66) > at > org.apache.camel.processor.SendProcessor.lambda$process$2(SendProcessor.java:188) > at > org.apache.camel.support.cache.DefaultProducerCache.doInAsyncProducer(DefaultProducerCache.java:317) > at > org.apache.camel.processor.SendProcessor.process(SendProcessor.java:187) > at > org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:395) > at > org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148) > at > org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60) > at > org.apache.camel.processor.Pipeline.process(Pipeline.java:147) > at > org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:312) > at > org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:207) > at > org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76) > at java.util.TimerThread.mainLoop(Timer.java:555) > at java.util.TimerThread.run(Timer.java:505) > Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed > connection during handshake > at > sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:994) > at > sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) > at > sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) > at > sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) > at > okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:336) > at > okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:300) > at > okhttp3.internal.connection.RealConnection.connect(RealConnection.jav
