[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710205#comment-14710205 ] Mayuresh Gharat commented on KAFKA-2351: Updated reviewboard https://reviews.apache.org/r/36652/diff/ against branch origin/trunk Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch, KAFKA-2351_2015-07-23_21:36:52.patch, KAFKA-2351_2015-08-13_13:10:05.patch, KAFKA-2351_2015-08-24_15:50:41.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14695859#comment-14695859 ] Mayuresh Gharat commented on KAFKA-2351: Updated reviewboard https://reviews.apache.org/r/36652/diff/ against branch origin/trunk Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch, KAFKA-2351_2015-07-23_21:36:52.patch, KAFKA-2351_2015-08-13_13:10:05.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660361#comment-14660361 ] Mayuresh Gharat commented on KAFKA-2351: Thanks [~junrao]. I will see if I can address the issue reported by [~sslavic] in a few days. If I can, I will upload a new patch else will do it in another ticket. Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch, KAFKA-2351_2015-07-23_21:36:52.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[
https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654927#comment-14654927
]
Stevo Slavic commented on KAFKA-2351:
-
Running single instance of Kafka broker from latest trunk I experienced this
not so successful controlled shutdown:
{noformat}
[2015-08-05 00:19:09,998] INFO [Offset Manager on Broker 0]: Removed 0 expired
offsets in 0 milliseconds. (kafka.server.OffsetManager)
^C[2015-08-05 00:23:09,144] INFO [Kafka Server 0], shutting down
(kafka.server.KafkaServer)
[2015-08-05 00:23:09,146] INFO [Kafka Server 0], Starting controlled shutdown
(kafka.server.KafkaServer)
[2015-08-05 00:23:09,155] ERROR [KafkaApi-0] error when handling request Name:
ControlledShutdownRequest; Version: 0; CorrelationId: 0; BrokerId: 0
(kafka.server.KafkaApis)
kafka.common.ControllerMovedException: Controller moved to another broker.
Aborting controlled shutdown
at
kafka.controller.KafkaController.shutdownBroker(KafkaController.scala:231)
at
kafka.server.KafkaApis.handleControlledShutdownRequest(KafkaApis.scala:146)
at kafka.server.KafkaApis.handle(KafkaApis.scala:63)
at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)
at java.lang.Thread.run(Thread.java:745)
[2015-08-05 00:23:09,156] INFO [Kafka Server 0], Remaining partitions to move:
(kafka.server.KafkaServer)
[2015-08-05 00:23:09,156] INFO [Kafka Server 0], Error code from controller: -1
(kafka.server.KafkaServer)
[2015-08-05 00:23:14,160] WARN [Kafka Server 0], Retrying controlled shutdown
after the previous attempt failed... (kafka.server.KafkaServer)
[2015-08-05 00:23:14,166] ERROR [KafkaApi-0] error when handling request Name:
ControlledShutdownRequest; Version: 0; CorrelationId: 1; BrokerId: 0
(kafka.server.KafkaApis)
kafka.common.ControllerMovedException: Controller moved to another broker.
Aborting controlled shutdown
at
kafka.controller.KafkaController.shutdownBroker(KafkaController.scala:231)
at
kafka.server.KafkaApis.handleControlledShutdownRequest(KafkaApis.scala:146)
at kafka.server.KafkaApis.handle(KafkaApis.scala:63)
at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)
at java.lang.Thread.run(Thread.java:745)
[2015-08-05 00:23:14,167] INFO [Kafka Server 0], Remaining partitions to move:
(kafka.server.KafkaServer)
[2015-08-05 00:23:14,167] INFO [Kafka Server 0], Error code from controller: -1
(kafka.server.KafkaServer)
[2015-08-05 00:23:19,169] WARN [Kafka Server 0], Retrying controlled shutdown
after the previous attempt failed... (kafka.server.KafkaServer)
[2015-08-05 00:23:19,172] ERROR [KafkaApi-0] error when handling request Name:
ControlledShutdownRequest; Version: 0; CorrelationId: 2; BrokerId: 0
(kafka.server.KafkaApis)
kafka.common.ControllerMovedException: Controller moved to another broker.
Aborting controlled shutdown
at
kafka.controller.KafkaController.shutdownBroker(KafkaController.scala:231)
at
kafka.server.KafkaApis.handleControlledShutdownRequest(KafkaApis.scala:146)
at kafka.server.KafkaApis.handle(KafkaApis.scala:63)
at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)
at java.lang.Thread.run(Thread.java:745)
[2015-08-05 00:23:19,173] INFO [Kafka Server 0], Remaining partitions to move:
(kafka.server.KafkaServer)
[2015-08-05 00:23:19,173] INFO [Kafka Server 0], Error code from controller: -1
(kafka.server.KafkaServer)
[2015-08-05 00:23:24,176] WARN [Kafka Server 0], Retrying controlled shutdown
after the previous attempt failed... (kafka.server.KafkaServer)
[2015-08-05 00:23:24,177] WARN [Kafka Server 0], Proceeding to do an unclean
shutdown as all the controlled shutdown attempts failed
(kafka.server.KafkaServer)
[2015-08-05 00:23:24,180] INFO [Socket Server on Broker 0], Shutting down
(kafka.network.SocketServer)
[2015-08-05 00:23:24,189] INFO [Socket Server on Broker 0], Shutdown completed
(kafka.network.SocketServer)
[2015-08-05 00:23:24,190] INFO [Kafka Request Handler on Broker 0], shutting
down (kafka.server.KafkaRequestHandlerPool)
[2015-08-05 00:23:24,193] INFO [Kafka Request Handler on Broker 0], shut down
completely (kafka.server.KafkaRequestHandlerPool)
[2015-08-05 00:23:24,196] INFO [Replica Manager on Broker 0]: Shutting down
(kafka.server.ReplicaManager)
[2015-08-05 00:23:24,196] INFO [ReplicaFetcherManager on broker 0] shutting
down (kafka.server.ReplicaFetcherManager)
[2015-08-05 00:23:24,197] INFO [ReplicaFetcherManager on broker 0] shutdown
completed (kafka.server.ReplicaFetcherManager)
[2015-08-05 00:23:24,197] INFO [ExpirationReaper-0], Shutting down
(kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2015-08-05 00:23:24,310] INFO [ExpirationReaper-0], Stopped
(kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2015-08-05 00:23:24,310] INFO
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14658749#comment-14658749 ] Mayuresh Gharat commented on KAFKA-2351: [~sslavic] I think this is different. Thanks for reporting this. Let me see if I can address this in the current patch. Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch, KAFKA-2351_2015-07-23_21:36:52.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659235#comment-14659235 ] Jun Rao commented on KAFKA-2351: [~mgharat], sorry the delay. Just replied on RB. Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch, KAFKA-2351_2015-07-23_21:36:52.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654691#comment-14654691 ] Mayuresh Gharat commented on KAFKA-2351: [~junrao] would you mind taking another look at this. I have replied to your comments on the RB. Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch, KAFKA-2351_2015-07-23_21:36:52.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644934#comment-14644934 ] Guozhang Wang commented on KAFKA-2351: -- [~mgharat] Could you change the status to Patch Available if you think it is ready for review again? [~junrao] will help reviewing it. Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch, KAFKA-2351_2015-07-23_21:36:52.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14641009#comment-14641009 ] Mayuresh Gharat commented on KAFKA-2351: Hi [~junrao], Thanks for the comments on the patch. I have replied to your concerns on the RB. Would you mind taking another look? I will upload a new patch accordingly. Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch, KAFKA-2351_2015-07-23_21:36:52.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639910#comment-14639910 ] Mayuresh Gharat commented on KAFKA-2351: Updated reviewboard https://reviews.apache.org/r/36652/diff/ against branch origin/trunk Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch, KAFKA-2351_2015-07-23_21:36:52.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638991#comment-14638991 ] Ismael Juma commented on KAFKA-2351: [~mgharat], did you see the discussion around exceptions in the review for https://issues.apache.org/jira/browse/KAFKA-2353 ? Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639097#comment-14639097 ] Mayuresh Gharat commented on KAFKA-2351: Hi [~ijuma], thanks a lot for pointing me towards this. I will think more on this and upload a new patch. Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638076#comment-14638076 ] Mayuresh Gharat commented on KAFKA-2351: [~junrao], [~guozhang] can you take a look at this? Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14635885#comment-14635885 ] Mayuresh Gharat commented on KAFKA-2351: Updated reviewboard https://reviews.apache.org/r/36652/diff/ against branch origin/trunk Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch, KAFKA-2351_2015-07-21_14:58:13.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2351) Brokers are having a problem shutting down correctly
[ https://issues.apache.org/jira/browse/KAFKA-2351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14635734#comment-14635734 ] Mayuresh Gharat commented on KAFKA-2351: Created reviewboard https://reviews.apache.org/r/36652/diff/ against branch origin/trunk Brokers are having a problem shutting down correctly Key: KAFKA-2351 URL: https://issues.apache.org/jira/browse/KAFKA-2351 Project: Kafka Issue Type: Bug Reporter: Mayuresh Gharat Assignee: Mayuresh Gharat Attachments: KAFKA-2351.patch The run() in Acceptor during shutdown might throw an exception that is not caught and it never reaches shutdownComplete due to which the latch is not counted down and the broker will not be able to shutdown. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
