[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14506527#comment-14506527 ] Dmitry Bugaychenko commented on KAFKA-2029: --- The problem is that the state is

[jira] [Issue Comment Deleted] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Comment: was deleted (was: The problem is that the state is distributed - not only

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14506528#comment-14506528 ] Dmitry Bugaychenko commented on KAFKA-2029: --- The problem is that the state is

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Attachment: KAFKA-2029.patch Improving controlled shutdown for rolling updates

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481465#comment-14481465 ] Dmitry Bugaychenko commented on KAFKA-2029: --- Created reviewboard

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481477#comment-14481477 ] Dmitry Bugaychenko commented on KAFKA-2029: --- Rebased on trunk. Improving

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381531#comment-14381531 ] Dmitry Bugaychenko commented on KAFKA-2029: --- Created reviewboard against branch

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Attachment: KAFKA-2029.patch Improving controlled shutdown for rolling updates

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381538#comment-14381538 ] Dmitry Bugaychenko commented on KAFKA-2029: --- Created reviewboard against branch

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Attachment: (was: KAFKA-2029.patch) Improving controlled shutdown for rolling

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Attachment: KAFKA-2029.patch Improving controlled shutdown for rolling updates

[jira] [Issue Comment Deleted] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Comment: was deleted (was: Created reviewboard against branch 0.8.1) Improving

[jira] [Comment Edited] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378433#comment-14378433 ] Dmitry Bugaychenko edited comment on KAFKA-2029 at 3/24/15 7:34 PM:

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378433#comment-14378433 ] Dmitry Bugaychenko commented on KAFKA-2029: --- We tried prioritization of

[jira] [Comment Edited] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378433#comment-14378433 ] Dmitry Bugaychenko edited comment on KAFKA-2029 at 3/24/15 7:35 PM:

[jira] [Commented] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371844#comment-14371844 ] Dmitry Bugaychenko commented on KAFKA-2029: --- You right about timeout - it was a

[jira] [Created] (KAFKA-2029) Improving controlled shutdown for rolling updates

Dmitry Bugaychenko created KAFKA-2029: - Summary: Improving controlled shutdown for rolling updates Key: KAFKA-2029 URL: https://issues.apache.org/jira/browse/KAFKA-2029 Project: Kafka

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Description: Controlled shutdown as implemented currently can cause numerous problems:

[jira] [Updated] (KAFKA-2029) Improving controlled shutdown for rolling updates

[ https://issues.apache.org/jira/browse/KAFKA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-2029: -- Description: Controlled shutdown as implemented currently can cause numerous problems:

[jira] [Commented] (KAFKA-1305) Controller can hang on controlled shutdown with auto leader balance enabled

[ https://issues.apache.org/jira/browse/KAFKA-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14363129#comment-14363129 ] Dmitry Bugaychenko commented on KAFKA-1305: --- Even with a fast dedicated channel

[jira] [Commented] (KAFKA-1305) Controller can hang on controlled shutdown with auto leader balance enabled

[ https://issues.apache.org/jira/browse/KAFKA-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14361155#comment-14361155 ] Dmitry Bugaychenko commented on KAFKA-1305: --- Yes, data loss is tolerated to some

[jira] [Commented] (KAFKA-1305) Controller can hang on controlled shutdown with auto leader balance enabled

[ https://issues.apache.org/jira/browse/KAFKA-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360253#comment-14360253 ] Dmitry Bugaychenko commented on KAFKA-1305: --- With a large controller queues size

[jira] [Commented] (KAFKA-1408) Kafk broker can not stop itself normaly after problems with connection to ZK

[ https://issues.apache.org/jira/browse/KAFKA-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14313765#comment-14313765 ] Dmitry Bugaychenko commented on KAFKA-1408: --- When topic deletion is disabled the

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074119#comment-14074119 ] Dmitry Bugaychenko commented on KAFKA-1414: --- I'm not a scala expert, but the

[jira] [Commented] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14069899#comment-14069899 ] Dmitry Bugaychenko commented on KAFKA-1539: --- Going to test power failure again

[jira] [Commented] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14070274#comment-14070274 ] Dmitry Bugaychenko commented on KAFKA-1539: --- With fileOutputStream.getFD.sync()

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14070555#comment-14070555 ] Dmitry Bugaychenko commented on KAFKA-1414: --- We would prefer the second

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14070642#comment-14070642 ] Dmitry Bugaychenko commented on KAFKA-1414: --- data dir and log dir seems more

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068333#comment-14068333 ] Dmitry Bugaychenko commented on KAFKA-1414: --- According our experience in

[jira] [Commented] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068010#comment-14068010 ] Dmitry Bugaychenko commented on KAFKA-1539: --- Digged the proble a bit more. It

[jira] [Comment Edited] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068010#comment-14068010 ] Dmitry Bugaychenko edited comment on KAFKA-1539 at 7/20/14 6:30 PM:

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068013#comment-14068013 ] Dmitry Bugaychenko commented on KAFKA-1414: --- In our use case inter-directory

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068015#comment-14068015 ] Dmitry Bugaychenko commented on KAFKA-1414: --- That is why in my initial proposal

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068020#comment-14068020 ] Dmitry Bugaychenko commented on KAFKA-1414: --- Separate thread pool per each dir

[jira] [Commented] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061724#comment-14061724 ] Dmitry Bugaychenko commented on KAFKA-1539: --- It looks like even after flush data

[jira] [Created] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

Dmitry Bugaychenko created KAFKA-1539: - Summary: Due to OS caching Kafka might loose offset files which causes full reset of data Key: KAFKA-1539 URL: https://issues.apache.org/jira/browse/KAFKA-1539

[jira] [Comment Edited] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060456#comment-14060456 ] Dmitry Bugaychenko edited comment on KAFKA-1539 at 7/14/14 8:58 AM:

[jira] [Commented] (KAFKA-1539) Due to OS caching Kafka might loose offset files which causes full reset of data

[ https://issues.apache.org/jira/browse/KAFKA-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14060456#comment-14060456 ] Dmitry Bugaychenko commented on KAFKA-1539: --- This is not about log files

[jira] [Commented] (KAFKA-1429) Yet another deadlock in controller shutdown

[ https://issues.apache.org/jira/browse/KAFKA-1429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985300#comment-13985300 ] Dmitry Bugaychenko commented on KAFKA-1429: --- Yes, we have

[jira] [Commented] (KAFKA-1429) Yet another deadlock in controller shutdown

[ https://issues.apache.org/jira/browse/KAFKA-1429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13984295#comment-13984295 ] Dmitry Bugaychenko commented on KAFKA-1429: --- Updated libs to 0.8.1.1, will see

[jira] [Created] (KAFKA-1429) Yet another deadlock in controller shutdown

Dmitry Bugaychenko created KAFKA-1429: - Summary: Yet another deadlock in controller shutdown Key: KAFKA-1429 URL: https://issues.apache.org/jira/browse/KAFKA-1429 Project: Kafka Issue

[jira] [Commented] (KAFKA-1414) Speedup broker startup after hard reset

[ https://issues.apache.org/jira/browse/KAFKA-1414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981025#comment-13981025 ] Dmitry Bugaychenko commented on KAFKA-1414: --- Actually similar improvement is

[jira] [Commented] (KAFKA-1407) Broker can not return to ISR because of BadVersionException

[ https://issues.apache.org/jira/browse/KAFKA-1407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13976597#comment-13976597 ] Dmitry Bugaychenko commented on KAFKA-1407: --- Checked out the logs in more

[jira] [Commented] (KAFKA-1408) Kafk broker can not stop itself normaly after problems with connection to ZK

[ https://issues.apache.org/jira/browse/KAFKA-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13976605#comment-13976605 ] Dmitry Bugaychenko commented on KAFKA-1408: --- We are using artefact from maven:

[jira] [Commented] (KAFKA-1408) Kafk broker can not stop itself normaly after problems with connection to ZK

[ https://issues.apache.org/jira/browse/KAFKA-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13976890#comment-13976890 ] Dmitry Bugaychenko commented on KAFKA-1408: --- Found this patch in the reviosion

[jira] [Created] (KAFKA-1407) Broker can not return to ISR because of BadVersionException

Dmitry Bugaychenko created KAFKA-1407: - Summary: Broker can not return to ISR because of BadVersionException Key: KAFKA-1407 URL: https://issues.apache.org/jira/browse/KAFKA-1407 Project: Kafka

[jira] [Created] (KAFKA-1408) Kafk broker can not stop itself normaly after problems with connection to ZK

Dmitry Bugaychenko created KAFKA-1408: - Summary: Kafk broker can not stop itself normaly after problems with connection to ZK Key: KAFKA-1408 URL: https://issues.apache.org/jira/browse/KAFKA-1408

[jira] [Updated] (KAFKA-1408) Kafk broker can not stop itself normaly after problems with connection to ZK

[ https://issues.apache.org/jira/browse/KAFKA-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Bugaychenko updated KAFKA-1408: -- Description: After getting to inconsistence state due to short netwrok failure broker