Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-19 Thread Jorg Heymans
Issue created here: https://issues.apache.org/jira/projects/KAFKA/issues/KAFKA-9202 On 2019/11/15 15:54:51, Jorg Heymans wrote: > Debugging into ConsoleConsumer.scala it eventually just calls this: > > val convertedBytes = deserializer.map(_.deserialize(topic, > nonNullBytes).toString. >

Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-15 Thread Jorg Heymans
Debugging into ConsoleConsumer.scala it eventually just calls this: val convertedBytes = deserializer.map(_.deserialize(topic, nonNullBytes).toString. getBytes(StandardCharsets.UTF_8)).getOrElse(nonNullBytes) See

Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-13 Thread Jorg Heymans
Hi, Sorry for the late reply, here is the command: kafka-console-consumer.cmd --consumer.config my-config.properties --bootstrap-server kafka-host:9443 --topic my-topic --value-deserializer my.BasicDeserializer --group my-console-group --property print.key=true my-config.properties only

Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-12 Thread M. Manna
HI, On Tue, 12 Nov 2019 at 14:37, Jorg Heymans wrote: > Thanks for helping debugging this. You can reproduce the issue using below > deserializer, and invoking kafka-console-consumer with > --value-deserializer=my.BasicDeserializer . As you will see, when the > consumer starts receiving

Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-12 Thread Jorg Heymans
Thanks for helping debugging this. You can reproduce the issue using below deserializer, and invoking kafka-console-consumer with --value-deserializer=my.BasicDeserializer . As you will see, when the consumer starts receiving messages only "SERDE WITHOUT HEADERS" is printed to the console.

Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-12 Thread M. Manna
HI again, On Tue, 12 Nov 2019 at 12:31, Jorg Heymans wrote: > Hi, > > The issue is not that i cannot get a custom deserializer working, it's > that the custom deserializer i provide implements the default method from > the Deserializer interface >

Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-12 Thread Jorg Heymans
Hi, The issue is not that i cannot get a custom deserializer working, it's that the custom deserializer i provide implements the default method from the Deserializer interface

Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-12 Thread M. Manna
Hi On Tue, 12 Nov 2019 at 09:53, Jorg Heymans wrote: > Indeed, i corrected the typo but now my deserializer class is not taken > into account at all and it goes back to the default deserializer. You can > verify this by putting a non-existent class and it still runs fine. > >

Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-12 Thread Jorg Heymans
Indeed, i corrected the typo but now my deserializer class is not taken into account at all and it goes back to the default deserializer. You can verify this by putting a non-existent class and it still runs fine. value.deserializer=does.not.exist Jorg On 2019/11/11 14:31:49, "M. Manna"

Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-11 Thread M. Manna
You have a typo - you mean deserializer Please try again. Regards, On Mon, 11 Nov 2019 at 14:28, Jorg Heymans wrote: > Don't think that option is available there, specifying > 'value.deserializer' in my consumer-config.properties file gives > > [2019-11-11 15:16:26,589] WARN The configuration

Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-11 Thread Jorg Heymans
Don't think that option is available there, specifying 'value.deserializer' in my consumer-config.properties file gives [2019-11-11 15:16:26,589] WARN The configuration 'value.serializer' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig) Does there

Re: kafka-console-consumer --value-deserializer with access to headers

2019-11-11 Thread M. Manna
Hi, On Mon, 11 Nov 2019 at 10:58, Jorg Heymans wrote: > Hi, > > I have created a class implementing Deserializer, providing an > implementation for > > public String deserialize(String topic, Headers headers, byte[] data) > > that does some conditional processing based on headers, and then

kafka-console-consumer --value-deserializer with access to headers

2019-11-11 Thread Jorg Heymans
Hi, I have created a class implementing Deserializer, providing an implementation for public String deserialize(String topic, Headers headers, byte[] data) that does some conditional processing based on headers, and then calls the other serde method public String deserialize(String topic,