Hi Omar Yeah the maven plugin idea is really good. But maybe we need both.
The maven plugin will only get you the options of the kafka version that was used for that release. With having a way for end users to set extra options then they could work against newer Kafka versions without Camel have to upgrade. And maybe also to proprietary options on Confluent. On Sun, Oct 13, 2019 at 2:31 PM Omar Al-Safi <omars...@gmail.com> wrote: > > Hi Claus, > > I also thought about this, however since Kafka producers/consumers options > increase and some of them may gets deprecated, I was thinking we can have a > maven plugin that generate these configurations for us without the need to > do it manually, it will generate using the configDef (e.g: > https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java#L555). > I did this for DBZ as I built a maven plugin that scrap all these config > and generate the camel configurations, you can take a look at my progress > here: > https://github.com/omarsmak/camel/blob/CAMEL-14009/components/camel-debezium/camel-debezium-maven-plugin/src/main/java/org/apache/camel/maven/TestMojo.java#L24. > Please take a look and share your thoughts about it. > > Regards, > Omar > > On Sun, 13 Oct 2019 at 14:12, Claus Ibsen <claus.ib...@gmail.com> wrote: > > > Hi > > > > Yeah this has come up a few times that Kafka is adding new options, or > > more particularly Confluent adds propriety options that they dont give > > back to the community. > > > > Can you create a JIRA ticket. > > > > Then we can look at adding a way to add custom options. You are of > > course welcome to try to implement this. We love contributions. > > https://github.com/apache/camel/blob/master/CONTRIBUTING.md > > > > On Tue, Oct 8, 2019 at 9:57 AM Erik Boye <erik.b...@statnett.no> wrote: > > > > > > Hi, > > > > > > Is there any way to configure custom Kafka producer/consumer properties > > on the Camel Kafka component/endpoint (without "breaking in")? > > > I specifically want to configure the Confluent Kafka property with key > > 'value.subject.name.strategy': > > https://docs.confluent.io/current/schema-registry/serializer-formatter.html > > > > > > Other Camel components (i.e Camel ActiveMQ; destination.consumer) are > > made extensible by providing users a way to set "additional properties". > > Why does not Camel Kafka provide that? Until schemaRegistryURL was added as > > a configurable property, we had to make our own Camel Kafka Component to > > make it work ("breaking in"). There should definitely be a way to set > > additional/custom properties on Kafka producers/consumers. > > > > > > Just have a look at the Camel Kafka documentation and do a text search > > for Confluent: > > https://camel.apache.org/components/latest/kafka-component.html ;-) > > > Having "native" support for schemas on Kafka records is definitely a > > good idea, and I am pretty sure Apache Kafka will move in that direction at > > some point.... > > > > > > Erik > > > > > > "This email with attachments is solely for the use of the individual or > > entity to which it is addressed. It may contain confidential or privileged > > information. If you are not the addressee, please notify the sender and > > delete this message and all attachments from your files." > > > > > > > > -- > > Claus Ibsen > > ----------------- > > http://davsclaus.com @davsclaus > > Camel in Action 2: https://www.manning.com/ibsen2 > > -- Claus Ibsen ----------------- http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
