[
https://issues.apache.org/jira/browse/KAFKA-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162964#comment-16162964
]
Seweryn Habdank-Wojewodzki edited comment on KAFKA-5861 at 9/12/17 1:50 PM:
----------------------------------------------------------------------------
I made further investigations. Thanks to inspiration from Ted Yu, I dumped
stack when java process was blocked.
I used "jcmd PID Thread.print" command (jstack is obsolete).
>From this stack I saw that indeed System.exit() blocks complete process.
So finally I modify my shutdown hook handler to such a code, where lock is very
primitive {{Object lock = new Object()}}:
{code:java}
final Thread mainThread = Thread.currentThread();
Runtime.getRuntime().addShutdownHook( new Thread( () -> {
synchronized ( lock ) {
LOG.info( "Got shutdown requests ..." );
streams.close( 5L, TimeUnit.SECONDS );
LOG.info( "Streamer is stopped." );
try {
// found at:
https://stackoverflow.com/questions/2921945/useful-example-of-a-shutdown-hook-in-java
mainThread.join( 1000 );
} catch ( InterruptedException e ) {
e.printStackTrace();
}
}
}
) );
{code}
This one seems to work and I can gracefully shutdown using kill PID command.
was (Author: habdank):
I made further investigations. Thanks to inspiration from Ted Yu, I dumped
stack when java process was blocked.
I used "jcmd PID Thread.print" command (jstack is obsolete).
>From this stack I saw that indeed System.exit() blocks complete process.
So finally I modify my shutdown hook handler to such a code, where lock is very
primitive {{Object lock = new Object()}}:
{code:java}
final Thread mainThread = Thread.currentThread();
Runtime.getRuntime().addShutdownHook( new Thread( () -> {
synchronized ( lock ) {
LOG.info( "Got shutdown requests ..." );
streams.close( 5L, TimeUnit.SECONDS );
LOG.info( "Streamer is stopped." );
try {
// found at:
https://stackoverflow.com/questions/2921945/useful-example-of-a-shutdown-hook-in-java
mainThread.join( 1000 );
} catch ( InterruptedException e ) {
e.printStackTrace();
}
}
}
) );
{code}
> KStream close( withTimeout ) - does not work under load conditions in the
> multi-threaded KStream application
> ------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-5861
> URL: https://issues.apache.org/jira/browse/KAFKA-5861
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.11.0.0
> Reporter: Seweryn Habdank-Wojewodzki
>
> Recently implemented close( withTimeout ) for streams does not work under
> load conditions in the multi-threaded KStream application.
> Where there are more consuming threads and there many messages in stream,
> then close ( withTimeout ) does not work.
> 1. Timeout is not respected at all and
> 2. application is hanging in some streaming chaos. Theoretically threads are
> working - they are busy with themselves, so the app cannot end, but they are
> not processing any further messages.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
