[ https://issues.apache.org/jira/browse/KAFKA-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162654#comment-16162654 ]
Seweryn Habdank-Wojewodzki edited comment on KAFKA-5861 at 9/12/17 8:22 AM: ---------------------------------------------------------------------------- Property: num.stream.threads=4; number of partitions in the topic = 10; replication factor = 2 I see the following, when I do not send message at all, I can stop the app by using kill PID. When I am sending messages, kill PID does not work at all. I need to use kill -9 PID. To be precise, I had made shutdown hook and then I see log "Got shutdown request ..." and nothing more. If I will send single message and then wait quite a long time (more than 30 or 60 sec) then it is probable kill PID will work. When I am sending steady couple like 2-5 messages per second mostly (>90%) kill PID does not work - I need to use kill -9 PID. When I am sending like 1000 or more messages per second, then kill PID does not work at all - again kill -9 PID was doing the job. But I suppose kill -9 PID is not the way to go, or? Core of my shutdown hook is below. The small comment - it is not important if I have System.exit(0) or not, as the procedure never comes to this point :-). Runtime.getRuntime().addShutdownHook( new Thread( () -> { LOG.info( "Got shutdown requests ..." ); synchronized ( this ) { streams.close( 5L, TimeUnit.SECONDS ); } LOG.info( "Restreamer is stopped." ); System.exit( 0 ); } ) ); was (Author: habdank): Property: num.stream.threads=4; number of partitions in the topic = 10; replication factor = 2 I see the following, when I do not send message at all, I can stop the app by using kill PID. When I am sending messages, kill PID does not work at all. I need to use kill -9 PID. To be precise, I had made shutdown hook and then I see log "Got shutdown request ..." and nothing more. If I will send single message and then wait quite a long time (more than 30 or 60 sec) then it is probable kill PID will work. When I am sending like 1000 or more messages per second, then kill PID does not work at all - again kill -9 PID was doing the job. But I suppose kill -9 PID is not the way to go, or? Core of my shutdown hoot is below. The small comment - it is not important if I have System.exit(0) or not, as the procedure never comes to this point :-). Runtime.getRuntime().addShutdownHook( new Thread( () -> { LOG.info( "Got shutdown requests ..." ); synchronized ( this ) { streams.close( 5L, TimeUnit.SECONDS ); } LOG.info( "Restreamer is stopped." ); System.exit( 0 ); } ) ); > KStream close( withTimeout ) - does not work under load conditions in the > multi-threaded KStream application > ------------------------------------------------------------------------------------------------------------ > > Key: KAFKA-5861 > URL: https://issues.apache.org/jira/browse/KAFKA-5861 > Project: Kafka > Issue Type: Bug > Components: streams > Affects Versions: 0.11.0.0 > Reporter: Seweryn Habdank-Wojewodzki > > Recently implemented close( withTimeout ) for streams does not work under > load conditions in the multi-threaded KStream application. > Where there are more consuming threads and there many messages in stream, > then close ( withTimeout ) does not work. > 1. Timeout is not respected at all and > 2. application is hanging in some streaming chaos. Theoretically threads are > working - they are busy with themselves, so the app cannot end, but they are > not processing any further messages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)