[GitHub] kafka pull request #3884: MINOR: various random minor fixes and improve Kafk...

2017-09-17 Thread mjsax
GitHub user mjsax opened a pull request:

https://github.com/apache/kafka/pull/3884

MINOR: various random minor fixes and improve KafkaConsumer JavaDocs



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mjsax/kafka 
minor-fixed-discoverd-via-exception-handling-investigation

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3884.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3884


commit 87c5cfa1634b546914cab2cbb4aa547ed020d763
Author: Matthias J. Sax 
Date:   2017-09-05T23:24:45Z

MINOR: various random minor fixes and improve KafkaConsumer JavaDocs




---


Re: Please add me to contributor list

2017-09-17 Thread 鄭紹志
Thanks !

Vito


On Mon, Sep 18, 2017 at 7:26 AM, Guozhang Wang  wrote:

> It's done. Cheers.
>
>
> Guozhang
>
> On Sat, Sep 16, 2017 at 4:52 PM, 鄭紹志  wrote:
>
> > My id: vitojeng
> >
> > Thanks, Guozhang.
> >
> >
> > Vito
> >
> >
> > On Sat, Sep 16, 2017 at 11:39 AM, Guozhang Wang 
> > wrote:
> >
> > > What's your apache id?
> > >
> > > On Sat, Sep 16, 2017 at 8:24 AM, 鄭紹志  wrote:
> > >
> > > > I want to work on some issue, please add me to contributor list in
> > JIRA.
> > > >
> > > > Also need to write permission in Confluence.
> > > >
> > > >
> > > > Thanks !
> > > > Vito
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>
>
>
> --
> -- Guozhang
>


Re: Please add me to contributor list

2017-09-17 Thread Guozhang Wang
It's done. Cheers.


Guozhang

On Sat, Sep 16, 2017 at 4:52 PM, 鄭紹志  wrote:

> My id: vitojeng
>
> Thanks, Guozhang.
>
>
> Vito
>
>
> On Sat, Sep 16, 2017 at 11:39 AM, Guozhang Wang 
> wrote:
>
> > What's your apache id?
> >
> > On Sat, Sep 16, 2017 at 8:24 AM, 鄭紹志  wrote:
> >
> > > I want to work on some issue, please add me to contributor list in
> JIRA.
> > >
> > > Also need to write permission in Confluence.
> > >
> > >
> > > Thanks !
> > > Vito
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>



-- 
-- Guozhang


[GitHub] kafka pull request #3880: KAFKA-5765 Move merge() from StreamsBuilder to KSt...

2017-09-17 Thread ConcurrencyPractitioner
GitHub user ConcurrencyPractitioner reopened a pull request:

https://github.com/apache/kafka/pull/3880

KAFKA-5765 Move merge() from StreamsBuilder to KStream

I have defined a {{merge()}} method to KStream. 
KStreamImpl overrides the {{merge()}} method.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ConcurrencyPractitioner/kafka trunk

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3880.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3880


commit 572f5bfa5a0e334c5b75a6e3353aa6f4b5b95f39
Author: Richard Yu 
Date:   2017-09-16T23:38:53Z

KAFKA-5765 diff --git 
a/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java 
b/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java

commit 71ac422ceb86e5c1d368a714b0af743eb2ad62eb
Author: Richard Yu 
Date:   2017-09-16T23:54:10Z

KAFKA-5765 Move merge() from StreamsBuilder to KStream




---


Please add me to contributor list in JIRA and Confluence

2017-09-17 Thread Jakub Scholz
Hi,

I would like to try to start contributing to the Kafka project, look at
some JIRAs and maybe raise some KIPs.

Could you please give me the JIRA rights to pick up some issues and the
rights to raise KIPs in Confluence Wiki? My username is scholzj for both
JIRA and Confluence.

Thanks & Regards
Jakub


[GitHub] kafka pull request #3883: KAFKA-5918: Fix minor typos and errors in the Kafk...

2017-09-17 Thread scholzj
GitHub user scholzj opened a pull request:

https://github.com/apache/kafka/pull/3883

KAFKA-5918: Fix minor typos and errors in the Kafka Streams turotial

I found several minor issues with the Kafka Streams tutorial:
* Some typos
  * "As shown above, it illustrate that the constructed ..." instead of "As 
shown above, it illustrate_s_ that the constructed ..."
  * "same as Pipe.java below" instead of "same as Pipe.java _above_"
  * Wrong class name in the `LineSplit` example
* Incorrect imports for the code examples
  * Missing `import org.apache.kafka.streams.kstream.KStream;` in 
`LineSplit` and `WordCount` example
* Unnecessary (and potentially confusing) split by whitespaces in the 
`WorkCount` class (the split into words happened already in `LineSplit`)


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/scholzj/kafka stream-tutorial-typos

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3883.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3883


commit 8facfd30948a4c866131c7554ded54bf4213293a
Author: Jakub Scholz 
Date:   2017-09-17T21:11:17Z

Fix some typos in the Kafka Streams tutorial




---


[jira] [Created] (KAFKA-5918) Fix minor typos and errors in the Kafka Streams turotial

2017-09-17 Thread Jakub Scholz (JIRA)
Jakub Scholz created KAFKA-5918:
---

 Summary: Fix minor typos and errors in the Kafka Streams turotial
 Key: KAFKA-5918
 URL: https://issues.apache.org/jira/browse/KAFKA-5918
 Project: Kafka
  Issue Type: Bug
  Components: documentation
Reporter: Jakub Scholz
Priority: Minor
 Fix For: 1.0.0


I found several minor issues with the Kafka Streams tutorial:
* Some typos
**  "As shown above, it illustrate that the constructed ..." instead of "As 
shown above, it illustrate_s_ that the constructed ..."
** "same as Pipe.java below" instead of "same as Pipe.java _above_"
** Wrong class name in the {{LineSplit}} example
* Incorrect imports for the code examples
** Missing {{import org.apache.kafka.streams.kstream.KStream;}} in 
{{LineSplit}} and {{WordCount}} example
* Unnecessary (and potentially confusing) split by whitespaces in the 
{{WorkCount}} class (the split into words happened already in {{LineSplit}})



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

2017-09-17 Thread Matthias J. Sax
Thanks for updating the KIP.

You are of course right, that we internally need access to
InternalStreamBuilder, but that should not be too hard and effectively
be an internal implementation detail.


Two more comments:

the new method should be

> KStream merge(KStream stream);

and not

>  KStream merge(KStream streams);

as in the KIP? The prefix `` is not required for non-static methods
and it should be singular (not plural) as parameter name?

Can you also add an explicit sentence, that the new method does not use
varargs anymore but a single KStream parameter (in contrast to the old
method). And mention that this is no limitation as calls to new merge()
can be chained.



Thanks a lot!

-Matthias



On 9/17/17 10:32 AM, Richard Yu wrote:
> Correction: When the current merge() method is called with multiple
> streams, a warning will be printed (or logged), but this should not hinder
> ability to read the log.
> There is a missing unchecked warning suppression for the old method.
> However, it is not high priority due to deprecation of the old merge()
> method.
> 
> 
> On Sun, Sep 17, 2017 at 9:37 AM, Richard Yu 
> wrote:
> 
>> With regards to Xavier's comment, this practice I do no think applies to
>> this PR. There is not much potential here for warnings to be thrown. Note
>> that in StreamsBuilder's merge, their is no 
>> @SuppressWarnings("unchecked")--indicating
>> that warnings is sparse, if not nonexistent.
>>
>>
>> On Sun, Sep 17, 2017 at 9:10 AM, Richard Yu 
>> wrote:
>>
>>> KIP-202 has been changed according to the conditions of your suggestion.
>>>
>>> On Sun, Sep 17, 2017 at 8:51 AM, Richard Yu 
>>> wrote:
>>>
 I added StreamsBuilder under the assumption that InternalStreamBuilder
 would be required to merge
 two streams. However, if that is not the case, then I would still need a
 couple of things:

 1) An InternalStreamBuilder instance to instantiate a new KStream

 2) The merge_name that the merged streams will be given

 3) Need access to the corresponding InternalStreamBuilder's
 InternalTopologyBuilder to add a processor (for the new KStreams)

 All these parameters are associated with InternalStreamsBuilder, thus it
 is essential towards merging the streams.
 We are left with three options (taking into account the restriction that
 InternalStreamsBuilder's reference scope is mostly limited to within the
 org.apache.kafka.streams.kstream.internals package):

 a) Find a way to pass InternalStreamsBuilder indirectly into the class.
 (using StreamsBuilder)

 b) Find the matching InternalStreamBuilder within the method that
 corresponds to the streams about to be merged.

 or c) Use the local InternalStreamsBuilder inherited from
 AbstractStream, assuming that it is the correct builder

 From your suggestion, that would mean using the c option I mentioned
 earlier. This choice of implementation works, but it could also include the
 risk that the local InternalStreamsBuilder might not be the correct one
 (just something one might want to keep in mind, since I will change it)

 On Sun, Sep 17, 2017 at 12:06 AM, Matthias J. Sax  wrote:

> Hi Richard,
>
> Thanks a lot for the KIP!
>
> I have three question:
>  - why is the new merge() method static?
>  - why does the new merge() method take StreamsBuilder as a parameter?
>  - did you think about Xavier's comment (see the JIRA in case you did
> not notice it yet) about varargs vs adding some overloads to merge
> stream?
>
> My personal take is that merge() should not be static and not take
> StreamsBuilder. The idea of the JIRA was to get a more natural API:
>
> // old
> KStream merged = StreamsBuilder.merge(stream1, stream2);
> // new
> KStream merge = stream1.merge(stream2);
>
>
> Having pointed out the second pattern, it should actually be fine to get
> rid of varargs in merger() at all, as users could chain multiple calls
> to merge() after each other:
>
> KStream multiMerged = stream1.merge(s2).merge(s3).merge(s4);
>
>
>
>
> -Matthias
>
> On 9/16/17 9:36 PM, Richard Yu wrote:
>> Hi,
>> Please take a look at:
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> 202+Move+merge%28%29+from+StreamsBuilder+to+KStream
>>
>> Thanks
>>
>
>

>>>
>>
> 



signature.asc
Description: OpenPGP digital signature


[GitHub] kafka pull request #3876: KAFKA-5896: Force Connect tasks to stop via thread...

2017-09-17 Thread 56quarters
GitHub user 56quarters reopened a pull request:

https://github.com/apache/kafka/pull/3876

KAFKA-5896: Force Connect tasks to stop via thread interruption

Interrupt the thread of Kafka Connect tasks that do not stop within
the timeout via `Worker::stopAndAwaitTasks()`. Previously tasks would
be asked to stop via setting a `stopping` flag. It was possible for
tasks to ignore this flag if they were, for example, waiting for
a lock or blocked on I/O.

This prevents issues where tasks may end up with multiple threads
all running and attempting to make progress when there should only
be a single thread running for that task at a time.

Fixes KAFKA-5896

/cc @rhauch @tedyu 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/smarter-travel-media/kafka force-task-stop

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3876.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3876


commit 31c879c1a1f0bd4f5999c021baca8e99e733ffe1
Author: Nick Pillitteri 
Date:   2017-09-13T14:54:40Z

Force Connect tasks to stop via thread interruption after a timeout

Interrupt the thread of Kafka Connect tasks that do not stop within
the timeout via Worker::stopAndAwaitTasks(). Previously tasks would
be asked to stop via setting a `stopping` flag. It was possible for
tasks to ignore this flag if they were, for example, waiting for
a lock or blocked on I/O.

This prevents issues where tasks may end up with multiple threads
all running and attempting to make progress when there should only
be a single thread running for that task at a time.

Fixes KAFKA-5896

commit ed3ef9c1f139cae4f09eefe1f66edc1c58c5ace6
Author: Nick Pillitteri 
Date:   2017-09-15T19:54:15Z

Rename per CR

commit afea834c708eae2b7d3dedbdeafed83197eaed94
Author: Nick Pillitteri 
Date:   2017-09-16T04:52:57Z

Rename per CR




---


[GitHub] kafka pull request #3876: KAFKA-5896: Force Connect tasks to stop via thread...

2017-09-17 Thread 56quarters
Github user 56quarters closed the pull request at:

https://github.com/apache/kafka/pull/3876


---


Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

2017-09-17 Thread Richard Yu
Correction: When the current merge() method is called with multiple
streams, a warning will be printed (or logged), but this should not hinder
ability to read the log.
There is a missing unchecked warning suppression for the old method.
However, it is not high priority due to deprecation of the old merge()
method.


On Sun, Sep 17, 2017 at 9:37 AM, Richard Yu 
wrote:

> With regards to Xavier's comment, this practice I do no think applies to
> this PR. There is not much potential here for warnings to be thrown. Note
> that in StreamsBuilder's merge, their is no 
> @SuppressWarnings("unchecked")--indicating
> that warnings is sparse, if not nonexistent.
>
>
> On Sun, Sep 17, 2017 at 9:10 AM, Richard Yu 
> wrote:
>
>> KIP-202 has been changed according to the conditions of your suggestion.
>>
>> On Sun, Sep 17, 2017 at 8:51 AM, Richard Yu 
>> wrote:
>>
>>> I added StreamsBuilder under the assumption that InternalStreamBuilder
>>> would be required to merge
>>> two streams. However, if that is not the case, then I would still need a
>>> couple of things:
>>>
>>> 1) An InternalStreamBuilder instance to instantiate a new KStream
>>>
>>> 2) The merge_name that the merged streams will be given
>>>
>>> 3) Need access to the corresponding InternalStreamBuilder's
>>> InternalTopologyBuilder to add a processor (for the new KStreams)
>>>
>>> All these parameters are associated with InternalStreamsBuilder, thus it
>>> is essential towards merging the streams.
>>> We are left with three options (taking into account the restriction that
>>> InternalStreamsBuilder's reference scope is mostly limited to within the
>>> org.apache.kafka.streams.kstream.internals package):
>>>
>>> a) Find a way to pass InternalStreamsBuilder indirectly into the class.
>>> (using StreamsBuilder)
>>>
>>> b) Find the matching InternalStreamBuilder within the method that
>>> corresponds to the streams about to be merged.
>>>
>>> or c) Use the local InternalStreamsBuilder inherited from
>>> AbstractStream, assuming that it is the correct builder
>>>
>>> From your suggestion, that would mean using the c option I mentioned
>>> earlier. This choice of implementation works, but it could also include the
>>> risk that the local InternalStreamsBuilder might not be the correct one
>>> (just something one might want to keep in mind, since I will change it)
>>>
>>> On Sun, Sep 17, 2017 at 12:06 AM, Matthias J. Sax >> > wrote:
>>>
 Hi Richard,

 Thanks a lot for the KIP!

 I have three question:
  - why is the new merge() method static?
  - why does the new merge() method take StreamsBuilder as a parameter?
  - did you think about Xavier's comment (see the JIRA in case you did
 not notice it yet) about varargs vs adding some overloads to merge
 stream?

 My personal take is that merge() should not be static and not take
 StreamsBuilder. The idea of the JIRA was to get a more natural API:

 // old
 KStream merged = StreamsBuilder.merge(stream1, stream2);
 // new
 KStream merge = stream1.merge(stream2);


 Having pointed out the second pattern, it should actually be fine to get
 rid of varargs in merger() at all, as users could chain multiple calls
 to merge() after each other:

 KStream multiMerged = stream1.merge(s2).merge(s3).merge(s4);




 -Matthias

 On 9/16/17 9:36 PM, Richard Yu wrote:
 > Hi,
 > Please take a look at:
 >
 > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
 > 202+Move+merge%28%29+from+StreamsBuilder+to+KStream
 >
 > Thanks
 >


>>>
>>
>


Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

2017-09-17 Thread Richard Yu
With regards to Xavier's comment, this practice I do no think applies to
this PR. There is not much potential here for warnings to be thrown. Note
that in StreamsBuilder's merge, their is no
@SuppressWarnings("unchecked")--indicating that warnings is sparse, if not
nonexistent.


On Sun, Sep 17, 2017 at 9:10 AM, Richard Yu 
wrote:

> KIP-202 has been changed according to the conditions of your suggestion.
>
> On Sun, Sep 17, 2017 at 8:51 AM, Richard Yu 
> wrote:
>
>> I added StreamsBuilder under the assumption that InternalStreamBuilder
>> would be required to merge
>> two streams. However, if that is not the case, then I would still need a
>> couple of things:
>>
>> 1) An InternalStreamBuilder instance to instantiate a new KStream
>>
>> 2) The merge_name that the merged streams will be given
>>
>> 3) Need access to the corresponding InternalStreamBuilder's
>> InternalTopologyBuilder to add a processor (for the new KStreams)
>>
>> All these parameters are associated with InternalStreamsBuilder, thus it
>> is essential towards merging the streams.
>> We are left with three options (taking into account the restriction that
>> InternalStreamsBuilder's reference scope is mostly limited to within the
>> org.apache.kafka.streams.kstream.internals package):
>>
>> a) Find a way to pass InternalStreamsBuilder indirectly into the class.
>> (using StreamsBuilder)
>>
>> b) Find the matching InternalStreamBuilder within the method that
>> corresponds to the streams about to be merged.
>>
>> or c) Use the local InternalStreamsBuilder inherited from AbstractStream,
>> assuming that it is the correct builder
>>
>> From your suggestion, that would mean using the c option I mentioned
>> earlier. This choice of implementation works, but it could also include the
>> risk that the local InternalStreamsBuilder might not be the correct one
>> (just something one might want to keep in mind, since I will change it)
>>
>> On Sun, Sep 17, 2017 at 12:06 AM, Matthias J. Sax 
>> wrote:
>>
>>> Hi Richard,
>>>
>>> Thanks a lot for the KIP!
>>>
>>> I have three question:
>>>  - why is the new merge() method static?
>>>  - why does the new merge() method take StreamsBuilder as a parameter?
>>>  - did you think about Xavier's comment (see the JIRA in case you did
>>> not notice it yet) about varargs vs adding some overloads to merge
>>> stream?
>>>
>>> My personal take is that merge() should not be static and not take
>>> StreamsBuilder. The idea of the JIRA was to get a more natural API:
>>>
>>> // old
>>> KStream merged = StreamsBuilder.merge(stream1, stream2);
>>> // new
>>> KStream merge = stream1.merge(stream2);
>>>
>>>
>>> Having pointed out the second pattern, it should actually be fine to get
>>> rid of varargs in merger() at all, as users could chain multiple calls
>>> to merge() after each other:
>>>
>>> KStream multiMerged = stream1.merge(s2).merge(s3).merge(s4);
>>>
>>>
>>>
>>>
>>> -Matthias
>>>
>>> On 9/16/17 9:36 PM, Richard Yu wrote:
>>> > Hi,
>>> > Please take a look at:
>>> >
>>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>> > 202+Move+merge%28%29+from+StreamsBuilder+to+KStream
>>> >
>>> > Thanks
>>> >
>>>
>>>
>>
>


[GitHub] kafka pull request #3882: MINOR: Add metric templates for sender/fetcher rat...

2017-09-17 Thread rajinisivaram
GitHub user rajinisivaram opened a pull request:

https://github.com/apache/kafka/pull/3882

MINOR: Add metric templates for sender/fetcher rate totals



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rajinisivaram/kafka 
MINOR-KAFKA-5738-metricstemplates

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3882.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3882


commit ac71252bf4a905df0a92e131eddca3024527ab9c
Author: Rajini Sivaram 
Date:   2017-09-17T16:14:48Z

MINOR: Add metric templates for sender/fetcher rate totals




---


Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

2017-09-17 Thread Richard Yu
KIP-202 has been changed according to the conditions of your suggestion.

On Sun, Sep 17, 2017 at 8:51 AM, Richard Yu 
wrote:

> I added StreamsBuilder under the assumption that InternalStreamBuilder
> would be required to merge
> two streams. However, if that is not the case, then I would still need a
> couple of things:
>
> 1) An InternalStreamBuilder instance to instantiate a new KStream
>
> 2) The merge_name that the merged streams will be given
>
> 3) Need access to the corresponding InternalStreamBuilder's
> InternalTopologyBuilder to add a processor (for the new KStreams)
>
> All these parameters are associated with InternalStreamsBuilder, thus it
> is essential towards merging the streams.
> We are left with three options (taking into account the restriction that
> InternalStreamsBuilder's reference scope is mostly limited to within the
> org.apache.kafka.streams.kstream.internals package):
>
> a) Find a way to pass InternalStreamsBuilder indirectly into the class.
> (using StreamsBuilder)
>
> b) Find the matching InternalStreamBuilder within the method that
> corresponds to the streams about to be merged.
>
> or c) Use the local InternalStreamsBuilder inherited from AbstractStream,
> assuming that it is the correct builder
>
> From your suggestion, that would mean using the c option I mentioned
> earlier. This choice of implementation works, but it could also include the
> risk that the local InternalStreamsBuilder might not be the correct one
> (just something one might want to keep in mind, since I will change it)
>
> On Sun, Sep 17, 2017 at 12:06 AM, Matthias J. Sax 
> wrote:
>
>> Hi Richard,
>>
>> Thanks a lot for the KIP!
>>
>> I have three question:
>>  - why is the new merge() method static?
>>  - why does the new merge() method take StreamsBuilder as a parameter?
>>  - did you think about Xavier's comment (see the JIRA in case you did
>> not notice it yet) about varargs vs adding some overloads to merge stream?
>>
>> My personal take is that merge() should not be static and not take
>> StreamsBuilder. The idea of the JIRA was to get a more natural API:
>>
>> // old
>> KStream merged = StreamsBuilder.merge(stream1, stream2);
>> // new
>> KStream merge = stream1.merge(stream2);
>>
>>
>> Having pointed out the second pattern, it should actually be fine to get
>> rid of varargs in merger() at all, as users could chain multiple calls
>> to merge() after each other:
>>
>> KStream multiMerged = stream1.merge(s2).merge(s3).merge(s4);
>>
>>
>>
>>
>> -Matthias
>>
>> On 9/16/17 9:36 PM, Richard Yu wrote:
>> > Hi,
>> > Please take a look at:
>> >
>> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> > 202+Move+merge%28%29+from+StreamsBuilder+to+KStream
>> >
>> > Thanks
>> >
>>
>>
>


[GitHub] kafka pull request #3880: KAFKA-5765 Move merge() from StreamsBuilder to KSt...

2017-09-17 Thread ConcurrencyPractitioner
Github user ConcurrencyPractitioner closed the pull request at:

https://github.com/apache/kafka/pull/3880


---


Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

2017-09-17 Thread Richard Yu
I added StreamsBuilder under the assumption that InternalStreamBuilder
would be required to merge
two streams. However, if that is not the case, then I would still need a
couple of things:

1) An InternalStreamBuilder instance to instantiate a new KStream

2) The merge_name that the merged streams will be given

3) Need access to the corresponding InternalStreamBuilder's
InternalTopologyBuilder to add a processor (for the new KStreams)

All these parameters are associated with InternalStreamsBuilder, thus it is
essential towards merging the streams.
We are left with three options (taking into account the restriction that
InternalStreamsBuilder's reference scope is mostly limited to within
the org.apache.kafka.streams.kstream.internals
package):

a) Find a way to pass InternalStreamsBuilder indirectly into the class.
(using StreamsBuilder)

b) Find the matching InternalStreamBuilder within the method that
corresponds to the streams about to be merged.

or c) Use the local InternalStreamsBuilder inherited from AbstractStream,
assuming that it is the correct builder

>From your suggestion, that would mean using the c option I mentioned
earlier. This choice of implementation works, but it could also include the
risk that the local InternalStreamsBuilder might not be the correct one
(just something one might want to keep in mind, since I will change it)

On Sun, Sep 17, 2017 at 12:06 AM, Matthias J. Sax 
wrote:

> Hi Richard,
>
> Thanks a lot for the KIP!
>
> I have three question:
>  - why is the new merge() method static?
>  - why does the new merge() method take StreamsBuilder as a parameter?
>  - did you think about Xavier's comment (see the JIRA in case you did
> not notice it yet) about varargs vs adding some overloads to merge stream?
>
> My personal take is that merge() should not be static and not take
> StreamsBuilder. The idea of the JIRA was to get a more natural API:
>
> // old
> KStream merged = StreamsBuilder.merge(stream1, stream2);
> // new
> KStream merge = stream1.merge(stream2);
>
>
> Having pointed out the second pattern, it should actually be fine to get
> rid of varargs in merger() at all, as users could chain multiple calls
> to merge() after each other:
>
> KStream multiMerged = stream1.merge(s2).merge(s3).merge(s4);
>
>
>
>
> -Matthias
>
> On 9/16/17 9:36 PM, Richard Yu wrote:
> > Hi,
> > Please take a look at:
> >
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 202+Move+merge%28%29+from+StreamsBuilder+to+KStream
> >
> > Thanks
> >
>
>


[GitHub] kafka pull request #3881: MINOR: Update powermock and enable its tests when ...

2017-09-17 Thread ijuma
GitHub user ijuma opened a pull request:

https://github.com/apache/kafka/pull/3881

MINOR: Update powermock and enable its tests when running with Java 9

Also fix WorkerTest to use the correct `Mock` annotations. 
`org.easymock.Mock`
is not supported by PowerMock 2.x.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ijuma/kafka kafka-5884-powermock-java

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3881.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3881


commit 3913864db02dde559c97808e2e5fc3a0833f38b6
Author: Ismael Juma 
Date:   2017-09-17T09:54:11Z

MINOR: Update powermock and enable its tests when running with Java 9




---


Re: [Discuss] KIP-202 Move merge() from StreamsBuilder to KStream

2017-09-17 Thread Matthias J. Sax
Hi Richard,

Thanks a lot for the KIP!

I have three question:
 - why is the new merge() method static?
 - why does the new merge() method take StreamsBuilder as a parameter?
 - did you think about Xavier's comment (see the JIRA in case you did
not notice it yet) about varargs vs adding some overloads to merge stream?

My personal take is that merge() should not be static and not take
StreamsBuilder. The idea of the JIRA was to get a more natural API:

// old
KStream merged = StreamsBuilder.merge(stream1, stream2);
// new
KStream merge = stream1.merge(stream2);


Having pointed out the second pattern, it should actually be fine to get
rid of varargs in merger() at all, as users could chain multiple calls
to merge() after each other:

KStream multiMerged = stream1.merge(s2).merge(s3).merge(s4);




-Matthias

On 9/16/17 9:36 PM, Richard Yu wrote:
> Hi,
> Please take a look at:
> 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 202+Move+merge%28%29+from+StreamsBuilder+to+KStream
> 
> Thanks
> 



signature.asc
Description: OpenPGP digital signature