Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Badai Aqrandista]

Hi all

Does anyone have time to review this PR?

I have created a PR for KIP-431 against the latest trunk:
https://github.com/apache/kafka/pull/9099

Regards
Badai

On Fri, Jul 10, 2020 at 2:57 PM huxi_2b  wrote:
>
> Badai,
>
> Replying in this thread. Two possible ways come up into my head:
>
> 1. CreateTime:1592475472398|key1|3|<0>|h1=v1,h2=v2|value1
> 2. CreateTime:1592475472398|key1|3|offset=0|h1=v1,h2=v2|value1
>
> I prefer the option #1 which could be accomplished by a complementary usage 
> description. What do you think?
>
> On 2020/06/21 13:39:36, Badai Aqrandista  wrote:
> > Excellent.>
> >
> > Would like to hear more feedback from others.>
> >
> > On Sat, Jun 20, 2020 at 1:27 AM David Jacot  wrote:>
> > >>
> > > Hi Badai,>
> > >>
> > > Thanks for your reply.>
> > >>
> > > 2. Yes, that makes sense.>
> > >>
> > > Best,>
> > > David>
> > >>
> > > On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista  
> > > wrote:>
> > >>
> > > > David>
> > > >>
> > > > Thank you for replying>
> > > >>
> > > > 1. It seems that `print.partition` is already implemented. Do you 
> > > > confirm?>
> > > > BADAI: Yes, you are correct. I have removed it from the KIP.>
> > > >>
> > > > 2. Will `null.literal` be only used when the value of the message>
> > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > today when the key or the value is empty. Shall we use "null" as>
> > > > a default instead of ""?>
> > > > BADAI: For any fields. Do you think this is useful?>
> > > >>
> > > > 3. Could we add a small example of the output in the KIP?>
> > > > BADAI: Yes, I have updated the KIP to add a couple of example.>
> > > >>
> > > > 4. When there are no headers, are we going to print something>
> > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > where there is no timestamp.>
> > > > BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.>
> > > >>
> > > > Thanks>
> > > > Badai>
> > > >>
> > > >>
> > > > On Thu, Jun 18, 2020 at 7:25 PM David Jacot  wrote:>
> > > > >>
> > > > > Hi Badai,>
> > > > >>
> > > > > Thanks for resuming this. I have few small comments:>
> > > > >>
> > > > > 1. It seems that `print.partition` is already implemented. Do you>
> > > > confirm?>
> > > > >>
> > > > > 2. Will `null.literal` be only used when the value of the message>
> > > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > > today when the key or the value is empty. Shall we use "null" as>
> > > > > a default instead of ""?>
> > > > >>
> > > > > 3. Could we add a small example of the output in the KIP?>
> > > > >>
> > > > > 4. When there are no headers, are we going to print something>
> > > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > > where there is no timestamp.>
> > > > >>
> > > > > Best,>
> > > > > David>
> > > > >>
> > > > > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista >
> > > > wrote:>
> > > > >>
> > > > > > Hi all,>
> > > > > >>
> > > > > > I have contacted Mateusz separately and he is ok for me to take 
> > > > > > over>
> > > > > > KIP-431:>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter>
> > > > > >>
> > > > > > I have updated it a bit. Can anyone give a quick look at it again 
> > > > > > and>
> > > > > > give me some feedback?>
> > > > > >>
> > > > > > This feature will be very helpful for people supporting Kafka in>
> > > > > > operations.>
> > > > > >>
> > > > > > If it is ready for a vote, please let me know.>
> > > > > >>
> > > > > > Thanks>
> > > > > > Badai>
> > > > > >>
> > > > > > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista 
> > > > > > >
> > > > > > wrote:>
> > > > > > >>
> > > > > > > Mateusz>
> > > > > > >>
> > > > > > > This KIP would be very useful for debugging. But the last 
> > > > > > > discussion>
> > > > > > > is in Feb 2019.>
> > > > > > >>
> > > > > > > Are you ok if I take over this KIP?>
> > > > > > >>
> > > > > > > -->
> > > > > > > Thanks,>
> > > > > > > Badai>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > > -->
> > > > > > Thanks,>
> > > > > > Badai>
> > > > > >>
> > > >>
> > > >>
> > > >>
> > > > -->
> > > > Thanks,>
> > > > Badai>
> > > >>
> >
> >
> >
> > -- >
> > Thanks,>
> > Badai>
> >



-- 
Thanks,
Badai

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Badai Aqrandista]

Hi all,

I just realised that KIP-431 introduces an incompatible change when an
undocumented property "print.partition" is set to "true". I found this
because of the system test failure. However, because the
incompatibility affects an undocumented property, I do not have
migration plan in this KIP.

And I have added the following to KIP-431 (
https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter
).

-
Compatibility, Deprecation, and Migration Plan

KIP-431 introduces incompatibility when "print.partition=true" (this
property exists in the code before KIP-431 but not documented). Before
KIP-431, "kafka-console-consumer" prints the partition as a number
after the value for example: "key1|value1|0". After this KIP,
"kafka-console-consumer" prints the partition number prefixed with
"Partition:" before the key (if printed) and value, for example:
"Partition:0|key1|value1" . Because this property was not documented,
no migration plan is implemented in KIP-431.

The other changes are backward compatible because they do not exist
before. Apart from "print.partition=true", if a user does not use any
new parameters, then the output of console consumer will look the same
as before.
-

What do you think?

Regards
Badai


On Wed, Aug 5, 2020 at 12:49 AM Badai Aqrandista  wrote:
>
> Hi all
>
> I have made additional changes to the PR as per review from David Jacot. 
> Please review.
>
> Regards
> Badai
>
> On Wed, Jul 29, 2020 at 11:17 PM Badai Aqrandista  wrote:
>>
>> Hi all
>>
>> I have created a PR for KIP-431 against the latest trunk:
>> https://github.com/apache/kafka/pull/9099
>>
>> Please review.
>>
>> Regards
>> Badai
>>
>> On Tue, Jul 21, 2020 at 2:13 AM Matthias J. Sax  wrote:
>> >
>> > Thanks Badai. LGTM.
>> >
>> > On 7/19/20 4:26 PM, Badai Aqrandista wrote:
>> > > Hi all
>> > >
>> > > I have made a small change to KIP-431 to make it clearer which one is
>> > > "Partition" and "Offset". Also I have moved key field to the back,
>> > > before the value:
>> > >
>> > > $ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic
>> > > test --from-beginning --property print.partition=true --property
>> > > print.key=true --property print.timestamp=true --property
>> > > print.offset=true --property print.headers=true --property
>> > > key.separator='|'
>> > >
>> > > CreateTime:1592475472398|Partition:0|Offset:3|h1:v1,h2:v2|key1|value1
>> > > CreateTime:1592475472456|Partition:0|Offset:4|NO_HEADERS|key2|value2
>> > >
>> > > Regards
>> > > Badai
>> > >
>> > > On Sun, Jun 21, 2020 at 11:39 PM Badai Aqrandista  
>> > > wrote:
>> > >>
>> > >> Excellent.
>> > >>
>> > >> Would like to hear more feedback from others.
>> > >>
>> > >> On Sat, Jun 20, 2020 at 1:27 AM David Jacot  wrote:
>> > >>>
>> > >>> Hi Badai,
>> > >>>
>> > >>> Thanks for your reply.
>> > >>>
>> > >>> 2. Yes, that makes sense.
>> > >>>
>> > >>> Best,
>> > >>> David
>> > >>>
>> > >>> On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista  
>> > >>> wrote:
>> > >>>
>> >  David
>> > 
>> >  Thank you for replying
>> > 
>> >  1. It seems that `print.partition` is already implemented. Do you 
>> >  confirm?
>> >  BADAI: Yes, you are correct. I have removed it from the KIP.
>> > 
>> >  2. Will `null.literal` be only used when the value of the message
>> >  is NULL or for any fields? Also, it seems that we print out "null"
>> >  today when the key or the value is empty. Shall we use "null" as
>> >  a default instead of ""?
>> >  BADAI: For any fields. Do you think this is useful?
>> > 
>> >  3. Could we add a small example of the output in the KIP?
>> >  BADAI: Yes, I have updated the KIP to add a couple of example.
>> > 
>> >  4. When there are no headers, are we going to print something
>> >  to indicate it to the user? For instance, we print out NO_TIMESTAMP
>> >  where there is no timestamp.
>> >  BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.
>> > 
>> >  Thanks
>> >  Badai
>> > 
>> > 
>> >  On Thu, Jun 18, 2020 at 7:25 PM David Jacot  
>> >  wrote:
>> > >
>> > > Hi Badai,
>> > >
>> > > Thanks for resuming this. I have few small comments:
>> > >
>> > > 1. It seems that `print.partition` is already implemented. Do you
>> >  confirm?
>> > >
>> > > 2. Will `null.literal` be only used when the value of the message
>> > > is NULL or for any fields? Also, it seems that we print out "null"
>> > > today when the key or the value is empty. Shall we use "null" as
>> > > a default instead of ""?
>> > >
>> > > 3. Could we add a small example of the output in the KIP?
>> > >
>> > > 4. When there are no headers, are we going to print something
>> > > to indicate it to the user? For instance, we print out NO_TIMESTAMP
>> > > where there is no timestamp.
>> > >
>> > > 

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Badai Aqrandista]

Hi all

I have made additional changes to the PR as per review from David Jacot.
Please review.

Regards
Badai

On Wed, Jul 29, 2020 at 11:17 PM Badai Aqrandista 
wrote:

> Hi all
>
> I have created a PR for KIP-431 against the latest trunk:
> https://github.com/apache/kafka/pull/9099
>
> Please review.
>
> Regards
> Badai
>
> On Tue, Jul 21, 2020 at 2:13 AM Matthias J. Sax  wrote:
> >
> > Thanks Badai. LGTM.
> >
> > On 7/19/20 4:26 PM, Badai Aqrandista wrote:
> > > Hi all
> > >
> > > I have made a small change to KIP-431 to make it clearer which one is
> > > "Partition" and "Offset". Also I have moved key field to the back,
> > > before the value:
> > >
> > > $ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic
> > > test --from-beginning --property print.partition=true --property
> > > print.key=true --property print.timestamp=true --property
> > > print.offset=true --property print.headers=true --property
> > > key.separator='|'
> > >
> > > CreateTime:1592475472398|Partition:0|Offset:3|h1:v1,h2:v2|key1|value1
> > > CreateTime:1592475472456|Partition:0|Offset:4|NO_HEADERS|key2|value2
> > >
> > > Regards
> > > Badai
> > >
> > > On Sun, Jun 21, 2020 at 11:39 PM Badai Aqrandista 
> wrote:
> > >>
> > >> Excellent.
> > >>
> > >> Would like to hear more feedback from others.
> > >>
> > >> On Sat, Jun 20, 2020 at 1:27 AM David Jacot 
> wrote:
> > >>>
> > >>> Hi Badai,
> > >>>
> > >>> Thanks for your reply.
> > >>>
> > >>> 2. Yes, that makes sense.
> > >>>
> > >>> Best,
> > >>> David
> > >>>
> > >>> On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista 
> wrote:
> > >>>
> >  David
> > 
> >  Thank you for replying
> > 
> >  1. It seems that `print.partition` is already implemented. Do you
> confirm?
> >  BADAI: Yes, you are correct. I have removed it from the KIP.
> > 
> >  2. Will `null.literal` be only used when the value of the message
> >  is NULL or for any fields? Also, it seems that we print out "null"
> >  today when the key or the value is empty. Shall we use "null" as
> >  a default instead of ""?
> >  BADAI: For any fields. Do you think this is useful?
> > 
> >  3. Could we add a small example of the output in the KIP?
> >  BADAI: Yes, I have updated the KIP to add a couple of example.
> > 
> >  4. When there are no headers, are we going to print something
> >  to indicate it to the user? For instance, we print out NO_TIMESTAMP
> >  where there is no timestamp.
> >  BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.
> > 
> >  Thanks
> >  Badai
> > 
> > 
> >  On Thu, Jun 18, 2020 at 7:25 PM David Jacot 
> wrote:
> > >
> > > Hi Badai,
> > >
> > > Thanks for resuming this. I have few small comments:
> > >
> > > 1. It seems that `print.partition` is already implemented. Do you
> >  confirm?
> > >
> > > 2. Will `null.literal` be only used when the value of the message
> > > is NULL or for any fields? Also, it seems that we print out "null"
> > > today when the key or the value is empty. Shall we use "null" as
> > > a default instead of ""?
> > >
> > > 3. Could we add a small example of the output in the KIP?
> > >
> > > 4. When there are no headers, are we going to print something
> > > to indicate it to the user? For instance, we print out NO_TIMESTAMP
> > > where there is no timestamp.
> > >
> > > Best,
> > > David
> > >
> > > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista <
> ba...@confluent.io>
> >  wrote:
> > >
> > >> Hi all,
> > >>
> > >> I have contacted Mateusz separately and he is ok for me to take
> over
> > >> KIP-431:
> > >>
> > >>
> > >>
> > 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter
> > >>
> > >> I have updated it a bit. Can anyone give a quick look at it again
> and
> > >> give me some feedback?
> > >>
> > >> This feature will be very helpful for people supporting Kafka in
> > >> operations.
> > >>
> > >> If it is ready for a vote, please let me know.
> > >>
> > >> Thanks
> > >> Badai
> > >>
> > >> On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista <
> ba...@confluent.io>
> > >> wrote:
> > >>>
> > >>> Mateusz
> > >>>
> > >>> This KIP would be very useful for debugging. But the last
> discussion
> > >>> is in Feb 2019.
> > >>>
> > >>> Are you ok if I take over this KIP?
> > >>>
> > >>> --
> > >>> Thanks,
> > >>> Badai
> > >>
> > >>
> > >>
> > >> --
> > >> Thanks,
> > >> Badai
> > >>
> > 
> > 
> > 
> >  --
> >  Thanks,
> >  Badai
> > 
> > >>
> > >>
> > >>
> > >> --
> > >> Thanks,
> > >> Badai
> > >
> > >
> > >
> >
>
>
> --
> Thanks,
> Badai
>


-- 
Thanks,
Badai

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Badai Aqrandista]

Hi all

I have created a PR for KIP-431 against the latest trunk:
https://github.com/apache/kafka/pull/9099

Please review.

Regards
Badai

On Tue, Jul 21, 2020 at 2:13 AM Matthias J. Sax  wrote:
>
> Thanks Badai. LGTM.
>
> On 7/19/20 4:26 PM, Badai Aqrandista wrote:
> > Hi all
> >
> > I have made a small change to KIP-431 to make it clearer which one is
> > "Partition" and "Offset". Also I have moved key field to the back,
> > before the value:
> >
> > $ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic
> > test --from-beginning --property print.partition=true --property
> > print.key=true --property print.timestamp=true --property
> > print.offset=true --property print.headers=true --property
> > key.separator='|'
> >
> > CreateTime:1592475472398|Partition:0|Offset:3|h1:v1,h2:v2|key1|value1
> > CreateTime:1592475472456|Partition:0|Offset:4|NO_HEADERS|key2|value2
> >
> > Regards
> > Badai
> >
> > On Sun, Jun 21, 2020 at 11:39 PM Badai Aqrandista  
> > wrote:
> >>
> >> Excellent.
> >>
> >> Would like to hear more feedback from others.
> >>
> >> On Sat, Jun 20, 2020 at 1:27 AM David Jacot  wrote:
> >>>
> >>> Hi Badai,
> >>>
> >>> Thanks for your reply.
> >>>
> >>> 2. Yes, that makes sense.
> >>>
> >>> Best,
> >>> David
> >>>
> >>> On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista  
> >>> wrote:
> >>>
>  David
> 
>  Thank you for replying
> 
>  1. It seems that `print.partition` is already implemented. Do you 
>  confirm?
>  BADAI: Yes, you are correct. I have removed it from the KIP.
> 
>  2. Will `null.literal` be only used when the value of the message
>  is NULL or for any fields? Also, it seems that we print out "null"
>  today when the key or the value is empty. Shall we use "null" as
>  a default instead of ""?
>  BADAI: For any fields. Do you think this is useful?
> 
>  3. Could we add a small example of the output in the KIP?
>  BADAI: Yes, I have updated the KIP to add a couple of example.
> 
>  4. When there are no headers, are we going to print something
>  to indicate it to the user? For instance, we print out NO_TIMESTAMP
>  where there is no timestamp.
>  BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.
> 
>  Thanks
>  Badai
> 
> 
>  On Thu, Jun 18, 2020 at 7:25 PM David Jacot  wrote:
> >
> > Hi Badai,
> >
> > Thanks for resuming this. I have few small comments:
> >
> > 1. It seems that `print.partition` is already implemented. Do you
>  confirm?
> >
> > 2. Will `null.literal` be only used when the value of the message
> > is NULL or for any fields? Also, it seems that we print out "null"
> > today when the key or the value is empty. Shall we use "null" as
> > a default instead of ""?
> >
> > 3. Could we add a small example of the output in the KIP?
> >
> > 4. When there are no headers, are we going to print something
> > to indicate it to the user? For instance, we print out NO_TIMESTAMP
> > where there is no timestamp.
> >
> > Best,
> > David
> >
> > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista 
>  wrote:
> >
> >> Hi all,
> >>
> >> I have contacted Mateusz separately and he is ok for me to take over
> >> KIP-431:
> >>
> >>
> >>
>  https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter
> >>
> >> I have updated it a bit. Can anyone give a quick look at it again and
> >> give me some feedback?
> >>
> >> This feature will be very helpful for people supporting Kafka in
> >> operations.
> >>
> >> If it is ready for a vote, please let me know.
> >>
> >> Thanks
> >> Badai
> >>
> >> On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista 
> >> wrote:
> >>>
> >>> Mateusz
> >>>
> >>> This KIP would be very useful for debugging. But the last discussion
> >>> is in Feb 2019.
> >>>
> >>> Are you ok if I take over this KIP?
> >>>
> >>> --
> >>> Thanks,
> >>> Badai
> >>
> >>
> >>
> >> --
> >> Thanks,
> >> Badai
> >>
> 
> 
> 
>  --
>  Thanks,
>  Badai
> 
> >>
> >>
> >>
> >> --
> >> Thanks,
> >> Badai
> >
> >
> >
>


-- 
Thanks,
Badai

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Matthias J. Sax]

Thanks Badai. LGTM.

On 7/19/20 4:26 PM, Badai Aqrandista wrote:
> Hi all
> 
> I have made a small change to KIP-431 to make it clearer which one is
> "Partition" and "Offset". Also I have moved key field to the back,
> before the value:
> 
> $ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic
> test --from-beginning --property print.partition=true --property
> print.key=true --property print.timestamp=true --property
> print.offset=true --property print.headers=true --property
> key.separator='|'
> 
> CreateTime:1592475472398|Partition:0|Offset:3|h1:v1,h2:v2|key1|value1
> CreateTime:1592475472456|Partition:0|Offset:4|NO_HEADERS|key2|value2
> 
> Regards
> Badai
> 
> On Sun, Jun 21, 2020 at 11:39 PM Badai Aqrandista  wrote:
>>
>> Excellent.
>>
>> Would like to hear more feedback from others.
>>
>> On Sat, Jun 20, 2020 at 1:27 AM David Jacot  wrote:
>>>
>>> Hi Badai,
>>>
>>> Thanks for your reply.
>>>
>>> 2. Yes, that makes sense.
>>>
>>> Best,
>>> David
>>>
>>> On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista  wrote:
>>>
 David

 Thank you for replying

 1. It seems that `print.partition` is already implemented. Do you confirm?
 BADAI: Yes, you are correct. I have removed it from the KIP.

 2. Will `null.literal` be only used when the value of the message
 is NULL or for any fields? Also, it seems that we print out "null"
 today when the key or the value is empty. Shall we use "null" as
 a default instead of ""?
 BADAI: For any fields. Do you think this is useful?

 3. Could we add a small example of the output in the KIP?
 BADAI: Yes, I have updated the KIP to add a couple of example.

 4. When there are no headers, are we going to print something
 to indicate it to the user? For instance, we print out NO_TIMESTAMP
 where there is no timestamp.
 BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.

 Thanks
 Badai


 On Thu, Jun 18, 2020 at 7:25 PM David Jacot  wrote:
>
> Hi Badai,
>
> Thanks for resuming this. I have few small comments:
>
> 1. It seems that `print.partition` is already implemented. Do you
 confirm?
>
> 2. Will `null.literal` be only used when the value of the message
> is NULL or for any fields? Also, it seems that we print out "null"
> today when the key or the value is empty. Shall we use "null" as
> a default instead of ""?
>
> 3. Could we add a small example of the output in the KIP?
>
> 4. When there are no headers, are we going to print something
> to indicate it to the user? For instance, we print out NO_TIMESTAMP
> where there is no timestamp.
>
> Best,
> David
>
> On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista 
 wrote:
>
>> Hi all,
>>
>> I have contacted Mateusz separately and he is ok for me to take over
>> KIP-431:
>>
>>
>>
 https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter
>>
>> I have updated it a bit. Can anyone give a quick look at it again and
>> give me some feedback?
>>
>> This feature will be very helpful for people supporting Kafka in
>> operations.
>>
>> If it is ready for a vote, please let me know.
>>
>> Thanks
>> Badai
>>
>> On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista 
>> wrote:
>>>
>>> Mateusz
>>>
>>> This KIP would be very useful for debugging. But the last discussion
>>> is in Feb 2019.
>>>
>>> Are you ok if I take over this KIP?
>>>
>>> --
>>> Thanks,
>>> Badai
>>
>>
>>
>> --
>> Thanks,
>> Badai
>>



 --
 Thanks,
 Badai

>>
>>
>>
>> --
>> Thanks,
>> Badai
> 
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Badai Aqrandista]

Hi all

I have made a small change to KIP-431 to make it clearer which one is
"Partition" and "Offset". Also I have moved key field to the back,
before the value:

$ kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic
test --from-beginning --property print.partition=true --property
print.key=true --property print.timestamp=true --property
print.offset=true --property print.headers=true --property
key.separator='|'

CreateTime:1592475472398|Partition:0|Offset:3|h1:v1,h2:v2|key1|value1
CreateTime:1592475472456|Partition:0|Offset:4|NO_HEADERS|key2|value2

Regards
Badai

On Sun, Jun 21, 2020 at 11:39 PM Badai Aqrandista  wrote:
>
> Excellent.
>
> Would like to hear more feedback from others.
>
> On Sat, Jun 20, 2020 at 1:27 AM David Jacot  wrote:
> >
> > Hi Badai,
> >
> > Thanks for your reply.
> >
> > 2. Yes, that makes sense.
> >
> > Best,
> > David
> >
> > On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista  wrote:
> >
> > > David
> > >
> > > Thank you for replying
> > >
> > > 1. It seems that `print.partition` is already implemented. Do you confirm?
> > > BADAI: Yes, you are correct. I have removed it from the KIP.
> > >
> > > 2. Will `null.literal` be only used when the value of the message
> > > is NULL or for any fields? Also, it seems that we print out "null"
> > > today when the key or the value is empty. Shall we use "null" as
> > > a default instead of ""?
> > > BADAI: For any fields. Do you think this is useful?
> > >
> > > 3. Could we add a small example of the output in the KIP?
> > > BADAI: Yes, I have updated the KIP to add a couple of example.
> > >
> > > 4. When there are no headers, are we going to print something
> > > to indicate it to the user? For instance, we print out NO_TIMESTAMP
> > > where there is no timestamp.
> > > BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.
> > >
> > > Thanks
> > > Badai
> > >
> > >
> > > On Thu, Jun 18, 2020 at 7:25 PM David Jacot  wrote:
> > > >
> > > > Hi Badai,
> > > >
> > > > Thanks for resuming this. I have few small comments:
> > > >
> > > > 1. It seems that `print.partition` is already implemented. Do you
> > > confirm?
> > > >
> > > > 2. Will `null.literal` be only used when the value of the message
> > > > is NULL or for any fields? Also, it seems that we print out "null"
> > > > today when the key or the value is empty. Shall we use "null" as
> > > > a default instead of ""?
> > > >
> > > > 3. Could we add a small example of the output in the KIP?
> > > >
> > > > 4. When there are no headers, are we going to print something
> > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP
> > > > where there is no timestamp.
> > > >
> > > > Best,
> > > > David
> > > >
> > > > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista 
> > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > I have contacted Mateusz separately and he is ok for me to take over
> > > > > KIP-431:
> > > > >
> > > > >
> > > > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter
> > > > >
> > > > > I have updated it a bit. Can anyone give a quick look at it again and
> > > > > give me some feedback?
> > > > >
> > > > > This feature will be very helpful for people supporting Kafka in
> > > > > operations.
> > > > >
> > > > > If it is ready for a vote, please let me know.
> > > > >
> > > > > Thanks
> > > > > Badai
> > > > >
> > > > > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista 
> > > > > wrote:
> > > > > >
> > > > > > Mateusz
> > > > > >
> > > > > > This KIP would be very useful for debugging. But the last discussion
> > > > > > is in Feb 2019.
> > > > > >
> > > > > > Are you ok if I take over this KIP?
> > > > > >
> > > > > > --
> > > > > > Thanks,
> > > > > > Badai
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > Badai
> > > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Badai
> > >
>
>
>
> --
> Thanks,
> Badai



-- 
Thanks,
Badai

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Badai Aqrandista]

Hi all

The vote for KIP-431 has passed with 3 binding and 1 non-binding +1s, and
no objections.

Thanks everyone for reviews and feedback.

Regards,
Badai

On Fri, Jul 10, 2020 at 5:22 PM Badai Aqrandista  wrote:
>
> Hu
>
> I would make it more readable and move the key next to the value like this:
>
> CreateTime:1592475472398|Partition:3|Offset:0|Headers:h1=v1,h2=v2|key1|value1
>
> What do you think?
>
> In future KIP, we'll add some formatting like David Jacot suggested.
>
> Regards
> Badai
>
> On Fri, Jul 10, 2020 at 2:57 PM huxi_2b  wrote:
> >
> > Badai,
> >
> > Replying in this thread. Two possible ways come up into my head:
> >
> > 1. CreateTime:1592475472398|key1|3|<0>|h1=v1,h2=v2|value1
> > 2. CreateTime:1592475472398|key1|3|offset=0|h1=v1,h2=v2|value1
> >
> > I prefer the option #1 which could be accomplished by a complementary usage 
> > description. What do you think?
> >
> > On 2020/06/21 13:39:36, Badai Aqrandista  wrote:
> > > Excellent.>
> > >
> > > Would like to hear more feedback from others.>
> > >
> > > On Sat, Jun 20, 2020 at 1:27 AM David Jacot  wrote:>
> > > >>
> > > > Hi Badai,>
> > > >>
> > > > Thanks for your reply.>
> > > >>
> > > > 2. Yes, that makes sense.>
> > > >>
> > > > Best,>
> > > > David>
> > > >>
> > > > On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista  
> > > > wrote:>
> > > >>
> > > > > David>
> > > > >>
> > > > > Thank you for replying>
> > > > >>
> > > > > 1. It seems that `print.partition` is already implemented. Do you 
> > > > > confirm?>
> > > > > BADAI: Yes, you are correct. I have removed it from the KIP.>
> > > > >>
> > > > > 2. Will `null.literal` be only used when the value of the message>
> > > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > > today when the key or the value is empty. Shall we use "null" as>
> > > > > a default instead of ""?>
> > > > > BADAI: For any fields. Do you think this is useful?>
> > > > >>
> > > > > 3. Could we add a small example of the output in the KIP?>
> > > > > BADAI: Yes, I have updated the KIP to add a couple of example.>
> > > > >>
> > > > > 4. When there are no headers, are we going to print something>
> > > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > > where there is no timestamp.>
> > > > > BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.>
> > > > >>
> > > > > Thanks>
> > > > > Badai>
> > > > >>
> > > > >>
> > > > > On Thu, Jun 18, 2020 at 7:25 PM David Jacot  
> > > > > wrote:>
> > > > > >>
> > > > > > Hi Badai,>
> > > > > >>
> > > > > > Thanks for resuming this. I have few small comments:>
> > > > > >>
> > > > > > 1. It seems that `print.partition` is already implemented. Do you>
> > > > > confirm?>
> > > > > >>
> > > > > > 2. Will `null.literal` be only used when the value of the message>
> > > > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > > > today when the key or the value is empty. Shall we use "null" as>
> > > > > > a default instead of ""?>
> > > > > >>
> > > > > > 3. Could we add a small example of the output in the KIP?>
> > > > > >>
> > > > > > 4. When there are no headers, are we going to print something>
> > > > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > > > where there is no timestamp.>
> > > > > >>
> > > > > > Best,>
> > > > > > David>
> > > > > >>
> > > > > > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista 
> > > > > > >
> > > > > wrote:>
> > > > > >>
> > > > > > > Hi all,>
> > > > > > >>
> > > > > > > I have contacted Mateusz separately and he is ok for me to take 
> > > > > > > over>
> > > > > > > KIP-431:>
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter>
> > > > > > >>
> > > > > > > I have updated it a bit. Can anyone give a quick look at it again 
> > > > > > > and>
> > > > > > > give me some feedback?>
> > > > > > >>
> > > > > > > This feature will be very helpful for people supporting Kafka in>
> > > > > > > operations.>
> > > > > > >>
> > > > > > > If it is ready for a vote, please let me know.>
> > > > > > >>
> > > > > > > Thanks>
> > > > > > > Badai>
> > > > > > >>
> > > > > > > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista 
> > > > > > > >
> > > > > > > wrote:>
> > > > > > > >>
> > > > > > > > Mateusz>
> > > > > > > >>
> > > > > > > > This KIP would be very useful for debugging. But the last 
> > > > > > > > discussion>
> > > > > > > > is in Feb 2019.>
> > > > > > > >>
> > > > > > > > Are you ok if I take over this KIP?>
> > > > > > > >>
> > > > > > > > -->
> > > > > > > > Thanks,>
> > > > > > > > Badai>
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > > -->
> > > > > > > Thanks,>
> > > > > > > Badai>
> > > > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > > -->
> > > > > Thanks,>
> > > > > Badai>
> > > > >>
> > >
> > >
> > >
> > > -- >
> > > Thanks,>
> > > Badai>
> > >
>
>
>

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Badai Aqrandista]

Hu

I would make it more readable and move the key next to the value like this:

CreateTime:1592475472398|Partition:3|Offset:0|Headers:h1=v1,h2=v2|key1|value1

What do you think?

In future KIP, we'll add some formatting like David Jacot suggested.

Regards
Badai

On Fri, Jul 10, 2020 at 2:57 PM huxi_2b  wrote:
>
> Badai,
>
> Replying in this thread. Two possible ways come up into my head:
>
> 1. CreateTime:1592475472398|key1|3|<0>|h1=v1,h2=v2|value1
> 2. CreateTime:1592475472398|key1|3|offset=0|h1=v1,h2=v2|value1
>
> I prefer the option #1 which could be accomplished by a complementary usage 
> description. What do you think?
>
> On 2020/06/21 13:39:36, Badai Aqrandista  wrote:
> > Excellent.>
> >
> > Would like to hear more feedback from others.>
> >
> > On Sat, Jun 20, 2020 at 1:27 AM David Jacot  wrote:>
> > >>
> > > Hi Badai,>
> > >>
> > > Thanks for your reply.>
> > >>
> > > 2. Yes, that makes sense.>
> > >>
> > > Best,>
> > > David>
> > >>
> > > On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista  
> > > wrote:>
> > >>
> > > > David>
> > > >>
> > > > Thank you for replying>
> > > >>
> > > > 1. It seems that `print.partition` is already implemented. Do you 
> > > > confirm?>
> > > > BADAI: Yes, you are correct. I have removed it from the KIP.>
> > > >>
> > > > 2. Will `null.literal` be only used when the value of the message>
> > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > today when the key or the value is empty. Shall we use "null" as>
> > > > a default instead of ""?>
> > > > BADAI: For any fields. Do you think this is useful?>
> > > >>
> > > > 3. Could we add a small example of the output in the KIP?>
> > > > BADAI: Yes, I have updated the KIP to add a couple of example.>
> > > >>
> > > > 4. When there are no headers, are we going to print something>
> > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > where there is no timestamp.>
> > > > BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.>
> > > >>
> > > > Thanks>
> > > > Badai>
> > > >>
> > > >>
> > > > On Thu, Jun 18, 2020 at 7:25 PM David Jacot  wrote:>
> > > > >>
> > > > > Hi Badai,>
> > > > >>
> > > > > Thanks for resuming this. I have few small comments:>
> > > > >>
> > > > > 1. It seems that `print.partition` is already implemented. Do you>
> > > > confirm?>
> > > > >>
> > > > > 2. Will `null.literal` be only used when the value of the message>
> > > > > is NULL or for any fields? Also, it seems that we print out "null">
> > > > > today when the key or the value is empty. Shall we use "null" as>
> > > > > a default instead of ""?>
> > > > >>
> > > > > 3. Could we add a small example of the output in the KIP?>
> > > > >>
> > > > > 4. When there are no headers, are we going to print something>
> > > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP>
> > > > > where there is no timestamp.>
> > > > >>
> > > > > Best,>
> > > > > David>
> > > > >>
> > > > > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista >
> > > > wrote:>
> > > > >>
> > > > > > Hi all,>
> > > > > >>
> > > > > > I have contacted Mateusz separately and he is ok for me to take 
> > > > > > over>
> > > > > > KIP-431:>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter>
> > > > > >>
> > > > > > I have updated it a bit. Can anyone give a quick look at it again 
> > > > > > and>
> > > > > > give me some feedback?>
> > > > > >>
> > > > > > This feature will be very helpful for people supporting Kafka in>
> > > > > > operations.>
> > > > > >>
> > > > > > If it is ready for a vote, please let me know.>
> > > > > >>
> > > > > > Thanks>
> > > > > > Badai>
> > > > > >>
> > > > > > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista 
> > > > > > >
> > > > > > wrote:>
> > > > > > >>
> > > > > > > Mateusz>
> > > > > > >>
> > > > > > > This KIP would be very useful for debugging. But the last 
> > > > > > > discussion>
> > > > > > > is in Feb 2019.>
> > > > > > >>
> > > > > > > Are you ok if I take over this KIP?>
> > > > > > >>
> > > > > > > -->
> > > > > > > Thanks,>
> > > > > > > Badai>
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > > -->
> > > > > > Thanks,>
> > > > > > Badai>
> > > > > >>
> > > >>
> > > >>
> > > >>
> > > > -->
> > > > Thanks,>
> > > > Badai>
> > > >>
> >
> >
> >
> > -- >
> > Thanks,>
> > Badai>
> >



-- 
Thanks,
Badai

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[huxi_2b]

Badai,

Replying in this thread. Two possible ways come up into my head:

1. CreateTime:1592475472398|key1|3|<0>|h1=v1,h2=v2|value1
2. CreateTime:1592475472398|key1|3|offset=0|h1=v1,h2=v2|value1

I prefer the option #1 which could be accomplished by a complementary usage 
description. What do you think?

On 2020/06/21 13:39:36, Badai Aqrandista  wrote: 
> Excellent.> 
> 
> Would like to hear more feedback from others.> 
> 
> On Sat, Jun 20, 2020 at 1:27 AM David Jacot  wrote:> 
> >> 
> > Hi Badai,> 
> >> 
> > Thanks for your reply.> 
> >> 
> > 2. Yes, that makes sense.> 
> >> 
> > Best,> 
> > David> 
> >> 
> > On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista  
> > wrote:> 
> >> 
> > > David> 
> > >> 
> > > Thank you for replying> 
> > >> 
> > > 1. It seems that `print.partition` is already implemented. Do you 
> > > confirm?> 
> > > BADAI: Yes, you are correct. I have removed it from the KIP.> 
> > >> 
> > > 2. Will `null.literal` be only used when the value of the message> 
> > > is NULL or for any fields? Also, it seems that we print out "null"> 
> > > today when the key or the value is empty. Shall we use "null" as> 
> > > a default instead of ""?> 
> > > BADAI: For any fields. Do you think this is useful?> 
> > >> 
> > > 3. Could we add a small example of the output in the KIP?> 
> > > BADAI: Yes, I have updated the KIP to add a couple of example.> 
> > >> 
> > > 4. When there are no headers, are we going to print something> 
> > > to indicate it to the user? For instance, we print out NO_TIMESTAMP> 
> > > where there is no timestamp.> 
> > > BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.> 
> > >> 
> > > Thanks> 
> > > Badai> 
> > >> 
> > >> 
> > > On Thu, Jun 18, 2020 at 7:25 PM David Jacot  wrote:> 
> > > >> 
> > > > Hi Badai,> 
> > > >> 
> > > > Thanks for resuming this. I have few small comments:> 
> > > >> 
> > > > 1. It seems that `print.partition` is already implemented. Do you> 
> > > confirm?> 
> > > >> 
> > > > 2. Will `null.literal` be only used when the value of the message> 
> > > > is NULL or for any fields? Also, it seems that we print out "null"> 
> > > > today when the key or the value is empty. Shall we use "null" as> 
> > > > a default instead of ""?> 
> > > >> 
> > > > 3. Could we add a small example of the output in the KIP?> 
> > > >> 
> > > > 4. When there are no headers, are we going to print something> 
> > > > to indicate it to the user? For instance, we print out NO_TIMESTAMP> 
> > > > where there is no timestamp.> 
> > > >> 
> > > > Best,> 
> > > > David> 
> > > >> 
> > > > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista > 
> > > wrote:> 
> > > >> 
> > > > > Hi all,> 
> > > > >> 
> > > > > I have contacted Mateusz separately and he is ok for me to take over> 
> > > > > KIP-431:> 
> > > > >> 
> > > > >> 
> > > > >> 
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter>
> > >  
> > > > >> 
> > > > > I have updated it a bit. Can anyone give a quick look at it again 
> > > > > and> 
> > > > > give me some feedback?> 
> > > > >> 
> > > > > This feature will be very helpful for people supporting Kafka in> 
> > > > > operations.> 
> > > > >> 
> > > > > If it is ready for a vote, please let me know.> 
> > > > >> 
> > > > > Thanks> 
> > > > > Badai> 
> > > > >> 
> > > > > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista 
> > > > > > 
> > > > > wrote:> 
> > > > > >> 
> > > > > > Mateusz> 
> > > > > >> 
> > > > > > This KIP would be very useful for debugging. But the last 
> > > > > > discussion> 
> > > > > > is in Feb 2019.> 
> > > > > >> 
> > > > > > Are you ok if I take over this KIP?> 
> > > > > >> 
> > > > > > --> 
> > > > > > Thanks,> 
> > > > > > Badai> 
> > > > >> 
> > > > >> 
> > > > >> 
> > > > > --> 
> > > > > Thanks,> 
> > > > > Badai> 
> > > > >> 
> > >> 
> > >> 
> > >> 
> > > --> 
> > > Thanks,> 
> > > Badai> 
> > >> 
> 
> 
> 
> -- > 
> Thanks,> 
> Badai> 
> 

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Badai Aqrandista]

Excellent.

Would like to hear more feedback from others.

On Sat, Jun 20, 2020 at 1:27 AM David Jacot  wrote:
>
> Hi Badai,
>
> Thanks for your reply.
>
> 2. Yes, that makes sense.
>
> Best,
> David
>
> On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista  wrote:
>
> > David
> >
> > Thank you for replying
> >
> > 1. It seems that `print.partition` is already implemented. Do you confirm?
> > BADAI: Yes, you are correct. I have removed it from the KIP.
> >
> > 2. Will `null.literal` be only used when the value of the message
> > is NULL or for any fields? Also, it seems that we print out "null"
> > today when the key or the value is empty. Shall we use "null" as
> > a default instead of ""?
> > BADAI: For any fields. Do you think this is useful?
> >
> > 3. Could we add a small example of the output in the KIP?
> > BADAI: Yes, I have updated the KIP to add a couple of example.
> >
> > 4. When there are no headers, are we going to print something
> > to indicate it to the user? For instance, we print out NO_TIMESTAMP
> > where there is no timestamp.
> > BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.
> >
> > Thanks
> > Badai
> >
> >
> > On Thu, Jun 18, 2020 at 7:25 PM David Jacot  wrote:
> > >
> > > Hi Badai,
> > >
> > > Thanks for resuming this. I have few small comments:
> > >
> > > 1. It seems that `print.partition` is already implemented. Do you
> > confirm?
> > >
> > > 2. Will `null.literal` be only used when the value of the message
> > > is NULL or for any fields? Also, it seems that we print out "null"
> > > today when the key or the value is empty. Shall we use "null" as
> > > a default instead of ""?
> > >
> > > 3. Could we add a small example of the output in the KIP?
> > >
> > > 4. When there are no headers, are we going to print something
> > > to indicate it to the user? For instance, we print out NO_TIMESTAMP
> > > where there is no timestamp.
> > >
> > > Best,
> > > David
> > >
> > > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista 
> > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I have contacted Mateusz separately and he is ok for me to take over
> > > > KIP-431:
> > > >
> > > >
> > > >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter
> > > >
> > > > I have updated it a bit. Can anyone give a quick look at it again and
> > > > give me some feedback?
> > > >
> > > > This feature will be very helpful for people supporting Kafka in
> > > > operations.
> > > >
> > > > If it is ready for a vote, please let me know.
> > > >
> > > > Thanks
> > > > Badai
> > > >
> > > > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista 
> > > > wrote:
> > > > >
> > > > > Mateusz
> > > > >
> > > > > This KIP would be very useful for debugging. But the last discussion
> > > > > is in Feb 2019.
> > > > >
> > > > > Are you ok if I take over this KIP?
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > Badai
> > > >
> > > >
> > > >
> > > > --
> > > > Thanks,
> > > > Badai
> > > >
> >
> >
> >
> > --
> > Thanks,
> > Badai
> >



-- 
Thanks,
Badai

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[David Jacot]

Hi Badai,

Thanks for your reply.

2. Yes, that makes sense.

Best,
David

On Thu, Jun 18, 2020 at 2:08 PM Badai Aqrandista  wrote:

> David
>
> Thank you for replying
>
> 1. It seems that `print.partition` is already implemented. Do you confirm?
> BADAI: Yes, you are correct. I have removed it from the KIP.
>
> 2. Will `null.literal` be only used when the value of the message
> is NULL or for any fields? Also, it seems that we print out "null"
> today when the key or the value is empty. Shall we use "null" as
> a default instead of ""?
> BADAI: For any fields. Do you think this is useful?
>
> 3. Could we add a small example of the output in the KIP?
> BADAI: Yes, I have updated the KIP to add a couple of example.
>
> 4. When there are no headers, are we going to print something
> to indicate it to the user? For instance, we print out NO_TIMESTAMP
> where there is no timestamp.
> BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.
>
> Thanks
> Badai
>
>
> On Thu, Jun 18, 2020 at 7:25 PM David Jacot  wrote:
> >
> > Hi Badai,
> >
> > Thanks for resuming this. I have few small comments:
> >
> > 1. It seems that `print.partition` is already implemented. Do you
> confirm?
> >
> > 2. Will `null.literal` be only used when the value of the message
> > is NULL or for any fields? Also, it seems that we print out "null"
> > today when the key or the value is empty. Shall we use "null" as
> > a default instead of ""?
> >
> > 3. Could we add a small example of the output in the KIP?
> >
> > 4. When there are no headers, are we going to print something
> > to indicate it to the user? For instance, we print out NO_TIMESTAMP
> > where there is no timestamp.
> >
> > Best,
> > David
> >
> > On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista 
> wrote:
> >
> > > Hi all,
> > >
> > > I have contacted Mateusz separately and he is ok for me to take over
> > > KIP-431:
> > >
> > >
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter
> > >
> > > I have updated it a bit. Can anyone give a quick look at it again and
> > > give me some feedback?
> > >
> > > This feature will be very helpful for people supporting Kafka in
> > > operations.
> > >
> > > If it is ready for a vote, please let me know.
> > >
> > > Thanks
> > > Badai
> > >
> > > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista 
> > > wrote:
> > > >
> > > > Mateusz
> > > >
> > > > This KIP would be very useful for debugging. But the last discussion
> > > > is in Feb 2019.
> > > >
> > > > Are you ok if I take over this KIP?
> > > >
> > > > --
> > > > Thanks,
> > > > Badai
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Badai
> > >
>
>
>
> --
> Thanks,
> Badai
>

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Badai Aqrandista]

David

Thank you for replying

1. It seems that `print.partition` is already implemented. Do you confirm?
BADAI: Yes, you are correct. I have removed it from the KIP.

2. Will `null.literal` be only used when the value of the message
is NULL or for any fields? Also, it seems that we print out "null"
today when the key or the value is empty. Shall we use "null" as
a default instead of ""?
BADAI: For any fields. Do you think this is useful?

3. Could we add a small example of the output in the KIP?
BADAI: Yes, I have updated the KIP to add a couple of example.

4. When there are no headers, are we going to print something
to indicate it to the user? For instance, we print out NO_TIMESTAMP
where there is no timestamp.
BADAI: Yes, good idea. I have updated the KIP to print NO_HEADERS.

Thanks
Badai


On Thu, Jun 18, 2020 at 7:25 PM David Jacot  wrote:
>
> Hi Badai,
>
> Thanks for resuming this. I have few small comments:
>
> 1. It seems that `print.partition` is already implemented. Do you confirm?
>
> 2. Will `null.literal` be only used when the value of the message
> is NULL or for any fields? Also, it seems that we print out "null"
> today when the key or the value is empty. Shall we use "null" as
> a default instead of ""?
>
> 3. Could we add a small example of the output in the KIP?
>
> 4. When there are no headers, are we going to print something
> to indicate it to the user? For instance, we print out NO_TIMESTAMP
> where there is no timestamp.
>
> Best,
> David
>
> On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista  wrote:
>
> > Hi all,
> >
> > I have contacted Mateusz separately and he is ok for me to take over
> > KIP-431:
> >
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter
> >
> > I have updated it a bit. Can anyone give a quick look at it again and
> > give me some feedback?
> >
> > This feature will be very helpful for people supporting Kafka in
> > operations.
> >
> > If it is ready for a vote, please let me know.
> >
> > Thanks
> > Badai
> >
> > On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista 
> > wrote:
> > >
> > > Mateusz
> > >
> > > This KIP would be very useful for debugging. But the last discussion
> > > is in Feb 2019.
> > >
> > > Are you ok if I take over this KIP?
> > >
> > > --
> > > Thanks,
> > > Badai
> >
> >
> >
> > --
> > Thanks,
> > Badai
> >



--
Thanks,
Badai

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[David Jacot]

Hi Badai,

Thanks for resuming this. I have few small comments:

1. It seems that `print.partition` is already implemented. Do you confirm?

2. Will `null.literal` be only used when the value of the message
is NULL or for any fields? Also, it seems that we print out "null"
today when the key or the value is empty. Shall we use "null" as
a default instead of ""?

3. Could we add a small example of the output in the KIP?

4. When there are no headers, are we going to print something
to indicate it to the user? For instance, we print out NO_TIMESTAMP
where there is no timestamp.

Best,
David

On Wed, Jun 17, 2020 at 4:53 PM Badai Aqrandista  wrote:

> Hi all,
>
> I have contacted Mateusz separately and he is ok for me to take over
> KIP-431:
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter
>
> I have updated it a bit. Can anyone give a quick look at it again and
> give me some feedback?
>
> This feature will be very helpful for people supporting Kafka in
> operations.
>
> If it is ready for a vote, please let me know.
>
> Thanks
> Badai
>
> On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista 
> wrote:
> >
> > Mateusz
> >
> > This KIP would be very useful for debugging. But the last discussion
> > is in Feb 2019.
> >
> > Are you ok if I take over this KIP?
> >
> > --
> > Thanks,
> > Badai
>
>
>
> --
> Thanks,
> Badai
>

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Badai Aqrandista]

Hi all,

I have contacted Mateusz separately and he is ok for me to take over KIP-431:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter

I have updated it a bit. Can anyone give a quick look at it again and
give me some feedback?

This feature will be very helpful for people supporting Kafka in operations.

If it is ready for a vote, please let me know.

Thanks
Badai

On Sat, Jun 13, 2020 at 10:59 PM Badai Aqrandista  wrote:
>
> Mateusz
>
> This KIP would be very useful for debugging. But the last discussion
> is in Feb 2019.
>
> Are you ok if I take over this KIP?
>
> --
> Thanks,
> Badai



-- 
Thanks,
Badai

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Badai Aqrandista]

Mateusz

This KIP would be very useful for debugging. But the last discussion
is in Feb 2019.

Are you ok if I take over this KIP?

-- 
Thanks,
Badai

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Colin McCabe]

+1, looks like a good improvement!

Best,
Colin

On Mon, Feb 18, 2019, at 14:00, Mateusz Zakarczemny wrote:
> Hi all,
> 
> I have created a KIP to support additional message fields in console
> consumer:
> KIP-431 - Support of printing additional ConsumerRecord fields in
> DefaultMessageFormatter
> 
> 
> The main purpose of the proposed change is to allow printing message
> offset, partition and headers in console consumer. Changes are backward
> compatible and impact only console consumer parameters.
> 
> PR: https://github.com/apache/kafka/pull/4807
> Jira ticket: KAFKA-6733 
> 
> I'm waiting for your feedback.
> 
> Regards,
> Mateusz Zakarczemny
> 

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Manikumar]

Thanks for the KIP. looks good to me.

On Sat, Feb 23, 2019 at 4:24 AM Gwen Shapira  wrote:

> Love it. Thanks.
>
> On Mon, Feb 18, 2019 at 2:00 PM Mateusz Zakarczemny
>  wrote:
> >
> > Hi all,
> >
> > I have created a KIP to support additional message fields in console
> > consumer:
> > KIP-431 - Support of printing additional ConsumerRecord fields in
> > DefaultMessageFormatter
> > <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-431%3A+Support+of+printing+additional+ConsumerRecord+fields+in+DefaultMessageFormatter
> >
> >
> > The main purpose of the proposed change is to allow printing message
> > offset, partition and headers in console consumer. Changes are backward
> > compatible and impact only console consumer parameters.
> >
> > PR: https://github.com/apache/kafka/pull/4807
> > Jira ticket: KAFKA-6733 <
> https://issues.apache.org/jira/browse/KAFKA-6733>
> >
> > I'm waiting for your feedback.
> >
> > Regards,
> > Mateusz Zakarczemny
>
>
>
> --
> Gwen Shapira
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter | blog
>

Re: [DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Gwen Shapira]

Love it. Thanks.

On Mon, Feb 18, 2019 at 2:00 PM Mateusz Zakarczemny
 wrote:
>
> Hi all,
>
> I have created a KIP to support additional message fields in console
> consumer:
> KIP-431 - Support of printing additional ConsumerRecord fields in
> DefaultMessageFormatter
> 
>
> The main purpose of the proposed change is to allow printing message
> offset, partition and headers in console consumer. Changes are backward
> compatible and impact only console consumer parameters.
>
> PR: https://github.com/apache/kafka/pull/4807
> Jira ticket: KAFKA-6733 
>
> I'm waiting for your feedback.
>
> Regards,
> Mateusz Zakarczemny



-- 
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog

[DISCUSS] KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatter

[Mateusz Zakarczemny]

Hi all,

I have created a KIP to support additional message fields in console
consumer:
KIP-431 - Support of printing additional ConsumerRecord fields in
DefaultMessageFormatter


The main purpose of the proposed change is to allow printing message
offset, partition and headers in console consumer. Changes are backward
compatible and impact only console consumer parameters.

PR: https://github.com/apache/kafka/pull/4807
Jira ticket: KAFKA-6733 

I'm waiting for your feedback.

Regards,
Mateusz Zakarczemny