[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[james chien (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16017189#comment-16017189
 ] 

james chien commented on KAFKA-4830:


[~mjsax] Okay, I will study about how to write it :)

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>Assignee: james chien
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[Matthias J. Sax (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016046#comment-16016046
 ] 

Matthias J. Sax commented on KAFKA-4830:


Thanks for working on this [~james.c]. As mentioned above, this is a public API 
change and thus we need a KIP: 
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals 
We can discuss the proposed API there. If you need any help writing a KIP -- it 
should not be too hard for this Jira -- just let us know.

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>Assignee: james chien
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[james chien (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16015486#comment-16015486
 ] 

james chien commented on KAFKA-4830:


[~mjsax] I just make a new PR. https://github.com/apache/kafka/pull/3085

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>Assignee: james chien
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16015474#comment-16015474
 ] 

ASF GitHub Bot commented on KAFKA-4830:
---

GitHub user jedichien opened a pull request:

https://github.com/apache/kafka/pull/3085

KAFKA-4830: Augment KStream.print() to allow users pass in extra parameters 
in the printed string

I extend `KStream#print()` to `KStream#print(KeyValueMapper)`.
So I add the following methods :
1. `KStream#print(KeyValueMapper)`
2. `KStream#print(KeyValueMapper, String streamName)`
3. `KStream#print(KeyValueMapper, Serde, Serde)`
4. `KStream#print(KeyValueMapper, Serde, Serde, String 
streamName)`


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jedichien/kafka KAFKA-4830

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3085.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3085


commit da4bcbaa90ed2107128914f7d6ca92add6e616db
Author: jameschien 
Date:   2017-04-27T08:00:20Z

change connection refused message from level DEBUG into WARN.

commit 1adfd44424f2760def0a648c96129243088ef044
Author: jameschien 
Date:   2017-05-02T08:26:04Z

replace KeyValuePrinter and KStreamForeach with KStreamPeek

commit 407b48b83d3acd8e7803bd32e93aa28e48c446a9
Author: jameschien 
Date:   2017-05-02T10:45:23Z

replace KeyValuePrinter and KStreamForeach with KStreamPeek

commit 9c3caa19724606e2e8982ea1c63ce8a480945e73
Author: jameschien 
Date:   2017-05-02T10:51:01Z

revert

commit 80dc9358962c85bf7d369dea5fa5e41965071585
Author: JamesChien 
Date:   2017-05-03T01:20:56Z

remove unused variable

commit 0777963ffc452e69eb468e546c7e4881ecff4cde
Author: jameschien 
Date:   2017-05-05T09:01:46Z

replace 'KStreamPeek#KStreamPeekProcessor' with PrintForeachAction

commit 65eddb227ddc003e71e1e79a2d378014d652566c
Author: jameschien 
Date:   2017-05-05T09:03:59Z

revert and will open new issue to discuss whether to keep this or not

commit 08f37fb99e16c701f293d59d9d12a8f039e9613e
Author: jameschien 
Date:   2017-05-05T09:05:00Z

Merge branch 'trunk' of https://github.com/jedichien/kafka into trunk

replace 'KStreamPeek#KStreamPeekProcessor' with PrintForeachAction

commit 3371b13aec71d80e097212820b0beef625948f37
Author: jameschien 
Date:   2017-05-08T10:07:04Z

add final and remove unnecessary code

commit bfbcced3969701e525d6109372ff6cbedbd4
Author: jameschien 
Date:   2017-05-08T10:13:50Z

add final to local variable

commit 62972990a84c4a77c7ca8c3eba6028ddb3f73990
Author: jameschien 
Date:   2017-05-17T01:34:34Z

add KStreamPrint to print data message with PrintForeachAction

commit 57784600657c71ed4736704bf07af8060e79b74f
Author: jameschien 
Date:   2017-05-17T01:45:09Z

add KStreamPrint to print data message with PrintForeachAction

commit 83101ff29ffe52735883e8b71f61176388724229
Author: JamesChien 
Date:   2017-05-18T09:14:08Z

Augment KStream.print() to allow users pass in extra parameters in the 
printed string. I extend print() to KStream.print(KeyValueMapper), KStream.print(KeyValueMapper, String streamName), 
KStream.print(KeyValueMapper, Serde, Serde) and 
KStream.print(KeyValueMapper, Serde, Serde, String 
streamName).




> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>Assignee: james chien
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[james chien (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16015110#comment-16015110
 ] 

james chien commented on KAFKA-4830:


I think if we do that then we will introduce a new API called 
`KStream#print(KeyValueMapper)`.

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>Assignee: james chien
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[james chien (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16015043#comment-16015043
 ] 

james chien commented on KAFKA-4830:


great, I will work on this :)

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[Matthias J. Sax (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16014715#comment-16014715
 ] 

Matthias J. Sax commented on KAFKA-4830:


[~james.c] KAFKA-4772 got merged. So feel free to work on this JIRA now :)

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[james chien (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15988028#comment-15988028
 ] 

james chien commented on KAFKA-4830:


yes, I want to work on that !

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[Matthias J. Sax (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15987050#comment-15987050
 ] 

Matthias J. Sax commented on KAFKA-4830:


This JIRA is still open yes. But it's blocked by KAFKA-4772 -- maybe you want 
to work on that? It's also not assigned yet.

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[james chien (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986208#comment-15986208
 ] 

james chien commented on KAFKA-4830:


the issue still exists?

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[Marc Juchli (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895048#comment-15895048
 ] 

Marc Juchli commented on KAFKA-4830:


[~mjsax] that works for me, will catch up on Monday with that.
[~guozhang] thanks for the feedback. I already prepared a PR (not submitted 
yet) as I missed the discussion on 
https://issues.apache.org/jira/browse/KAFKA-4772. In this case I'll tackle the 
other one first.

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[Guozhang Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894842#comment-15894842
 ] 

Guozhang Wang commented on KAFKA-4830:
--

[~mjuchli] As for this JIRA itself, the goal is to allow users to customize the 
output format even with its own pass-in parameters instead of restricting with

{code}
"[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
{code}

However the prefix of "[streamName]" will still be preserved. I.e. we can add a 
new function in `KStream` as you mentioned. As for "KStreamPrinter", as 
Matthias mentioned, it is better to first cleanup this class as an special impl 
of `KStreamPeek` and then this special impl can be extended with a `mapper` 
parameter to implement the above API.

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[Matthias J. Sax (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894780#comment-15894780
 ] 

Matthias J. Sax commented on KAFKA-4830:


Thank for tackling this!

Two comments: (1) this JIRA is marked as "blocked by KAFKA-4772", so it would 
be good if you would do the other JIRA first. (2) this is a public API change 
and requires a KIP 
(https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals) 
-- thus, I would suggest to prepare a KIP and start to DISCUSS thread to talk 
about the details.

Would this work for you?

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>  Labels: needs-kip, newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper)}} :
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-4830) Augment KStream.print() to allow users pass in extra parameters in the printed string

[Marc Juchli (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894483#comment-15894483
 ] 

Marc Juchli commented on KAFKA-4830:


I'd like to tackle this, but beforehand I have a question. 
I see that your mentioned line of code is part of {{KeyValuePrinter}}, and now 
I am unsure whether this was just an example.
If my understanding is correct, what was suggested with 
{{KStream.print(KeyValueMapper)}} is meant to be an extension of 
{{KStream}} in order to let the user decide how the output of the K,V shall 
look like. However, this rises my concerns as we wouldn't allow to set a 
{{streamName}} with this function definition.
Therefore, my approach would be to extend KStream with {{void print(final 
String streamName, final KeyValueMapper mapper)}}.
Furthermore, as {{KStreamImpl.print}} creates an instance of `KeyValuePrinter`, 
this class would have to be changed too accordingly. Or is that meant be left 
open to the user?

> Augment KStream.print() to allow users pass in extra parameters in the 
> printed string
> -
>
> Key: KAFKA-4830
> URL: https://issues.apache.org/jira/browse/KAFKA-4830
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Reporter: Guozhang Wang
>  Labels: newbie
>
> Today {{KStream.print}} use the hard-coded result string as:
> {code}
> "[" + this.streamName + "]: " + keyToPrint + " , " + valueToPrint
> {code}
> And some users are asking to augment this so that they can customize the 
> output string as {{KStream.print(KeyValueMapper):
> {code}
> "[" + this.streamName + "]: " + mapper.apply(keyToPrint, valueToPrint)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)