[jira] [Commented] (KAFKA-5818) KafkaStreams state transitions not correct

2017-09-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151516#comment-16151516
 ] 

ASF GitHub Bot commented on KAFKA-5818:
---

GitHub user mjsax opened a pull request:

https://github.com/apache/kafka/pull/3779

KAFKA-5818: KafkaStreams state transitions not correct



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mjsax/kafka 
kafka-5818-kafkaStreams-state-transition-01101

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3779.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3779


commit 50ae6bcfd36dd9aa98f808c476dfdeb9fd8655e3
Author: Matthias J. Sax 
Date:   2017-09-02T14:49:12Z

KAFKA-5818: KafkaStreams state transitions not correct




> KafkaStreams state transitions not correct
> --
>
> Key: KAFKA-5818
> URL: https://issues.apache.org/jira/browse/KAFKA-5818
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.11.0.1, 1.0.0
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
> Fix For: 0.11.0.1, 1.0.0
>
>
> There is a race condition revealed by failing test 
> {{KafkaStreamsTest#testCannotStartTwise}}. It fails with:
> {noformat}
> java.lang.Exception: Unexpected exception, 
> expected but 
> was
> Caused by: java.lang.IllegalThreadStateException
>   at java.lang.Thread.start(Thread.java:705)
>   at org.apache.kafka.streams.KafkaStreams.start(KafkaStreams.java:590)
>   at 
> org.apache.kafka.streams.KafkaStreamsTest.testCannotStartTwice(KafkaStreamsTest.java:251)
> {noformat}
> The race condition is a follows:
> 1) test calls {{KafkaStreams#start()}} for the first time and state transits 
> from CREATED -> RUNNING
> 2) First poll triggers a rebalance and {{StreamThread}} put {{KafkaStreams}} 
> into state REBALANCING
> 3) before REBALANCING completes, the main test thread calls 
> {{KafkaStream#start()}} again. As current state is REBALANCING, the 
> transition to RUNNING is valid and {{start()}} does not fail with 
> {{IllegalStateException}} but resumes. When it tries to start internal 
> streams, we get {{IllegalThreadStateException}} as thread are already running.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-5818) KafkaStreams state transitions not correct

2017-09-02 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151431#comment-16151431
 ] 

ASF GitHub Bot commented on KAFKA-5818:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/3775


> KafkaStreams state transitions not correct
> --
>
> Key: KAFKA-5818
> URL: https://issues.apache.org/jira/browse/KAFKA-5818
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.11.0.1, 1.0.0
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
> Fix For: 0.11.0.1, 1.0.0
>
>
> There is a race condition revealed by failing test 
> {{KafkaStreamsTest#testCannotStartTwise}}. It fails with:
> {noformat}
> java.lang.Exception: Unexpected exception, 
> expected but 
> was
> Caused by: java.lang.IllegalThreadStateException
>   at java.lang.Thread.start(Thread.java:705)
>   at org.apache.kafka.streams.KafkaStreams.start(KafkaStreams.java:590)
>   at 
> org.apache.kafka.streams.KafkaStreamsTest.testCannotStartTwice(KafkaStreamsTest.java:251)
> {noformat}
> The race condition is a follows:
> 1) test calls {{KafkaStreams#start()}} for the first time and state transits 
> from CREATED -> RUNNING
> 2) First poll triggers a rebalance and {{StreamThread}} put {{KafkaStreams}} 
> into state REBALANCING
> 3) before REBALANCING completes, the main test thread calls 
> {{KafkaStream#start()}} again. As current state is REBALANCING, the 
> transition to RUNNING is valid and {{start()}} does not fail with 
> {{IllegalStateException}} but resumes. When it tries to start internal 
> streams, we get {{IllegalThreadStateException}} as thread are already running.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-5818) KafkaStreams state transitions not correct

2017-08-31 Thread Ted Yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149980#comment-16149980
 ] 

Ted Yu commented on KAFKA-5818:
---

{code}
public synchronized boolean isRunningAndNotRebalancing() {
// we do not need to grab stateLock since it is a single read
return state == State.RUNNING;
{code}
Does that mean the synchronized keyword can be removed from line 208 ?

> KafkaStreams state transitions not correct
> --
>
> Key: KAFKA-5818
> URL: https://issues.apache.org/jira/browse/KAFKA-5818
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.11.0.1, 1.0.0
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
> Fix For: 0.11.0.1, 1.0.0
>
>
> There is a race condition revealed by failing test 
> {{KafkaStreamsTest#testCannotStartTwise}}. It fails with:
> {noformat}
> java.lang.Exception: Unexpected exception, 
> expected but 
> was
> Caused by: java.lang.IllegalThreadStateException
>   at java.lang.Thread.start(Thread.java:705)
>   at org.apache.kafka.streams.KafkaStreams.start(KafkaStreams.java:590)
>   at 
> org.apache.kafka.streams.KafkaStreamsTest.testCannotStartTwice(KafkaStreamsTest.java:251)
> {noformat}
> The race condition is a follows:
> 1) test calls {{KafkaStreams#start()}} for the first time and state transits 
> from CREATED -> RUNNING
> 2) First poll triggers a rebalance and {{StreamThread}} put {{KafkaStreams}} 
> into state REBALANCING
> 3) before REBALANCING completes, the main test thread calls 
> {{KafkaStream#start()}} again. As current state is REBALANCING, the 
> transition to RUNNING is valid and {{start()}} does not fail with 
> {{IllegalStateException}} but resumes. When it tries to start internal 
> streams, we get {{IllegalThreadStateException}} as thread are already running.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-5818) KafkaStreams state transitions not correct

2017-08-31 Thread Guozhang Wang (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149975#comment-16149975
 ] 

Guozhang Wang commented on KAFKA-5818:
--

Yes, because state is volatile, we only need to synchronize the block that 
read-write the variable; for single read operation we do not need to 
synchronize it.

> KafkaStreams state transitions not correct
> --
>
> Key: KAFKA-5818
> URL: https://issues.apache.org/jira/browse/KAFKA-5818
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.11.0.1, 1.0.0
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
> Fix For: 0.11.0.1, 1.0.0
>
>
> There is a race condition revealed by failing test 
> {{KafkaStreamsTest#testCannotStartTwise}}. It fails with:
> {noformat}
> java.lang.Exception: Unexpected exception, 
> expected but 
> was
> Caused by: java.lang.IllegalThreadStateException
>   at java.lang.Thread.start(Thread.java:705)
>   at org.apache.kafka.streams.KafkaStreams.start(KafkaStreams.java:590)
>   at 
> org.apache.kafka.streams.KafkaStreamsTest.testCannotStartTwice(KafkaStreamsTest.java:251)
> {noformat}
> The race condition is a follows:
> 1) test calls {{KafkaStreams#start()}} for the first time and state transits 
> from CREATED -> RUNNING
> 2) First poll triggers a rebalance and {{StreamThread}} put {{KafkaStreams}} 
> into state REBALANCING
> 3) before REBALANCING completes, the main test thread calls 
> {{KafkaStream#start()}} again. As current state is REBALANCING, the 
> transition to RUNNING is valid and {{start()}} does not fail with 
> {{IllegalStateException}} but resumes. When it tries to start internal 
> streams, we get {{IllegalThreadStateException}} as thread are already running.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-5818) KafkaStreams state transitions not correct

2017-08-31 Thread Ted Yu (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149786#comment-16149786
 ] 

Ted Yu commented on KAFKA-5818:
---

Looking at StreamThread, in the call chain run() -> runLoop() -> runOnce() , 
there is statement accessing state (line 753) without synchronized keyword on 
any of the method:
{code}
if (state == State.PARTITIONS_ASSIGNED) {
{code}
Is this intentional ?

> KafkaStreams state transitions not correct
> --
>
> Key: KAFKA-5818
> URL: https://issues.apache.org/jira/browse/KAFKA-5818
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.11.0.1, 1.0.0
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
> Fix For: 0.11.0.1, 1.0.0
>
>
> There is a race condition revealed by failing test 
> {{KafkaStreamsTest#testCannotStartTwise}}. It fails with:
> {noformat}
> java.lang.Exception: Unexpected exception, 
> expected but 
> was
> Caused by: java.lang.IllegalThreadStateException
>   at java.lang.Thread.start(Thread.java:705)
>   at org.apache.kafka.streams.KafkaStreams.start(KafkaStreams.java:590)
>   at 
> org.apache.kafka.streams.KafkaStreamsTest.testCannotStartTwice(KafkaStreamsTest.java:251)
> {noformat}
> The race condition is a follows:
> 1) test calls {{KafkaStreams#start()}} for the first time and state transits 
> from CREATED -> RUNNING
> 2) First poll triggers a rebalance and {{StreamThread}} put {{KafkaStreams}} 
> into state REBALANCING
> 3) before REBALANCING completes, the main test thread calls 
> {{KafkaStream#start()}} again. As current state is REBALANCING, the 
> transition to RUNNING is valid and {{start()}} does not fail with 
> {{IllegalStateException}} but resumes. When it tries to start internal 
> streams, we get {{IllegalThreadStateException}} as thread are already running.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-5818) KafkaStreams state transitions not correct

2017-08-31 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149726#comment-16149726
 ] 

ASF GitHub Bot commented on KAFKA-5818:
---

GitHub user mjsax opened a pull request:

https://github.com/apache/kafka/pull/3775

KAFKA-5818: KafkaStreams state transitions not correct

- need to check that state is CRATED at startup
- some minor test cleanup

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mjsax/kafka 
kafka-5818-kafkaStreams-state-transition

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3775.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3775


commit 0ab4a2a15bf895ea7fdbfc476897dafa0717a1ae
Author: Matthias J. Sax 
Date:   2017-08-31T22:38:48Z

KAFKA-5818: KafkaStreams state transitions not correct




> KafkaStreams state transitions not correct
> --
>
> Key: KAFKA-5818
> URL: https://issues.apache.org/jira/browse/KAFKA-5818
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 0.11.0.1, 1.0.0
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
>
> There is a race condition revealed by failing test 
> {{KafkaStreamsTest#testCannotStartTwise}}. It fails with:
> {noformat}
> java.lang.Exception: Unexpected exception, 
> expected but 
> was
> Caused by: java.lang.IllegalThreadStateException
>   at java.lang.Thread.start(Thread.java:705)
>   at org.apache.kafka.streams.KafkaStreams.start(KafkaStreams.java:590)
>   at 
> org.apache.kafka.streams.KafkaStreamsTest.testCannotStartTwice(KafkaStreamsTest.java:251)
> {noformat}
> The race condition is a follows:
> 1) test calls {{KafkaStreams#start()}} for the first time and state transits 
> from CREATED -> RUNNING
> 2) First poll triggers a rebalance and {{StreamThread}} put {{KafkaStreams}} 
> into state REBALANCING
> 3) before REBALANCING completes, the main test thread calls 
> {{KafkaStream#start()}} again. As current state is REBALANCING, the 
> transition to RUNNING is valid and {{start()}} does not fail with 
> {{IllegalStateException}} but resumes. When it tries to start internal 
> streams, we get {{IllegalThreadStateException}} as thread are already running.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)