[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)
[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14625428#comment-14625428 ] Dong Lin commented on KAFKA-2241: - Updated reviewboard https://reviews.apache.org/r/34965/diff/ against branch origin/trunk > AbstractFetcherThread.shutdown() should not block on > ReadableByteChannel.read(buffer) > - > > Key: KAFKA-2241 > URL: https://issues.apache.org/jira/browse/KAFKA-2241 > Project: Kafka > Issue Type: Bug >Reporter: Dong Lin >Assignee: Dong Lin > Labels: quotas > Attachments: KAFKA-2241.patch, KAFKA-2241_2015-06-03_15:30:35.patch, > KAFKA-2241_2015-07-09_15:35:49.patch, KAFKA-2241_2015-07-13_13:30:07.patch, > KAFKA-2241_2015-07-13_14:51:42.patch, client.java, server.java > > > This is likely a bug from Java. This affects Kafka and here is the patch to > fix it. > Here is the description of the bug. By description of SocketChannel in Java 7 > Documentation. If another thread interrupts the current thread while the read > operation is in progress, the it should closes the channel and throw > ClosedByInterruptException. However, we find that interrupting the thread > will not unblock the channel immediately. Instead, it waits for response or > socket timeout before throwing an exception. > This will cause problem in the following scenario. Suppose one > console_consumer_1 is reading from a topic, and due to quota delay or > whatever reason, it block on channel.read(buffer). At this moment, another > console_consumer_2 joins and triggers rebalance at console_consumer_1. But > consumer_1 will block waiting on the channel.read before it can release > partition ownership, causing consumer_2 to fail after a number of failed > attempts to obtain partition ownership. > In other words, AbstractFetcherThread.shutdown() is not guaranteed to > shutdown due to this bug. > The problem is confirmed with Java 1.7 and java 1.6. To check it by yourself, > you can use the attached server.java and client.java -- start the server > before the client and see if client unblock after interruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)
[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14625284#comment-14625284 ] Dong Lin commented on KAFKA-2241: - Updated reviewboard https://reviews.apache.org/r/34965/diff/ against branch origin/trunk > AbstractFetcherThread.shutdown() should not block on > ReadableByteChannel.read(buffer) > - > > Key: KAFKA-2241 > URL: https://issues.apache.org/jira/browse/KAFKA-2241 > Project: Kafka > Issue Type: Bug >Reporter: Dong Lin >Assignee: Dong Lin > Labels: quotas > Attachments: KAFKA-2241.patch, KAFKA-2241_2015-06-03_15:30:35.patch, > KAFKA-2241_2015-07-09_15:35:49.patch, KAFKA-2241_2015-07-13_13:30:07.patch, > client.java, server.java > > > This is likely a bug from Java. This affects Kafka and here is the patch to > fix it. > Here is the description of the bug. By description of SocketChannel in Java 7 > Documentation. If another thread interrupts the current thread while the read > operation is in progress, the it should closes the channel and throw > ClosedByInterruptException. However, we find that interrupting the thread > will not unblock the channel immediately. Instead, it waits for response or > socket timeout before throwing an exception. > This will cause problem in the following scenario. Suppose one > console_consumer_1 is reading from a topic, and due to quota delay or > whatever reason, it block on channel.read(buffer). At this moment, another > console_consumer_2 joins and triggers rebalance at console_consumer_1. But > consumer_1 will block waiting on the channel.read before it can release > partition ownership, causing consumer_2 to fail after a number of failed > attempts to obtain partition ownership. > In other words, AbstractFetcherThread.shutdown() is not guaranteed to > shutdown due to this bug. > The problem is confirmed with Java 1.7 and java 1.6. To check it by yourself, > you can use the attached server.java and client.java -- start the server > before the client and see if client unblock after interruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)
[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621396#comment-14621396 ] Dong Lin commented on KAFKA-2241: - Updated reviewboard https://reviews.apache.org/r/34965/diff/ against branch origin/trunk > AbstractFetcherThread.shutdown() should not block on > ReadableByteChannel.read(buffer) > - > > Key: KAFKA-2241 > URL: https://issues.apache.org/jira/browse/KAFKA-2241 > Project: Kafka > Issue Type: Bug >Reporter: Dong Lin >Assignee: Dong Lin > Labels: quotas > Attachments: KAFKA-2241.patch, KAFKA-2241_2015-06-03_15:30:35.patch, > KAFKA-2241_2015-07-09_15:35:49.patch, client.java, server.java > > > This is likely a bug from Java. This affects Kafka and here is the patch to > fix it. > Here is the description of the bug. By description of SocketChannel in Java 7 > Documentation. If another thread interrupts the current thread while the read > operation is in progress, the it should closes the channel and throw > ClosedByInterruptException. However, we find that interrupting the thread > will not unblock the channel immediately. Instead, it waits for response or > socket timeout before throwing an exception. > This will cause problem in the following scenario. Suppose one > console_consumer_1 is reading from a topic, and due to quota delay or > whatever reason, it block on channel.read(buffer). At this moment, another > console_consumer_2 joins and triggers rebalance at console_consumer_1. But > consumer_1 will block waiting on the channel.read before it can release > partition ownership, causing consumer_2 to fail after a number of failed > attempts to obtain partition ownership. > In other words, AbstractFetcherThread.shutdown() is not guaranteed to > shutdown due to this bug. > The problem is confirmed with Java 1.7 and java 1.6. To check it by yourself, > you can use the attached server.java and client.java -- start the server > before the client and see if client unblock after interruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)
[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14615627#comment-14615627 ] Dong Lin commented on KAFKA-2241: - Hi [~junrao]], could you please review this patch when you have time? > AbstractFetcherThread.shutdown() should not block on > ReadableByteChannel.read(buffer) > - > > Key: KAFKA-2241 > URL: https://issues.apache.org/jira/browse/KAFKA-2241 > Project: Kafka > Issue Type: Bug >Reporter: Dong Lin >Assignee: Dong Lin > Labels: quotas > Attachments: KAFKA-2241.patch, KAFKA-2241_2015-06-03_15:30:35.patch, > client.java, server.java > > > This is likely a bug from Java. This affects Kafka and here is the patch to > fix it. > Here is the description of the bug. By description of SocketChannel in Java 7 > Documentation. If another thread interrupts the current thread while the read > operation is in progress, the it should closes the channel and throw > ClosedByInterruptException. However, we find that interrupting the thread > will not unblock the channel immediately. Instead, it waits for response or > socket timeout before throwing an exception. > This will cause problem in the following scenario. Suppose one > console_consumer_1 is reading from a topic, and due to quota delay or > whatever reason, it block on channel.read(buffer). At this moment, another > console_consumer_2 joins and triggers rebalance at console_consumer_1. But > consumer_1 will block waiting on the channel.read before it can release > partition ownership, causing consumer_2 to fail after a number of failed > attempts to obtain partition ownership. > In other words, AbstractFetcherThread.shutdown() is not guaranteed to > shutdown due to this bug. > The problem is confirmed with Java 1.7 and java 1.6. To check it by yourself, > you can use the attached server.java and client.java -- start the server > before the client and see if client unblock after interruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)
[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584381#comment-14584381 ] Dong Lin commented on KAFKA-2241: - [~junrao] No, this bug is fixed in java 1.8.0. Can you let me know if we should drop this ticket? > AbstractFetcherThread.shutdown() should not block on > ReadableByteChannel.read(buffer) > - > > Key: KAFKA-2241 > URL: https://issues.apache.org/jira/browse/KAFKA-2241 > Project: Kafka > Issue Type: Bug >Reporter: Dong Lin >Assignee: Dong Lin > Labels: quotas > Attachments: KAFKA-2241.patch, KAFKA-2241_2015-06-03_15:30:35.patch, > client.java, server.java > > > This is likely a bug from Java. This affects Kafka and here is the patch to > fix it. > Here is the description of the bug. By description of SocketChannel in Java 7 > Documentation. If another thread interrupts the current thread while the read > operation is in progress, the it should closes the channel and throw > ClosedByInterruptException. However, we find that interrupting the thread > will not unblock the channel immediately. Instead, it waits for response or > socket timeout before throwing an exception. > This will cause problem in the following scenario. Suppose one > console_consumer_1 is reading from a topic, and due to quota delay or > whatever reason, it block on channel.read(buffer). At this moment, another > console_consumer_2 joins and triggers rebalance at console_consumer_1. But > consumer_1 will block waiting on the channel.read before it can release > partition ownership, causing consumer_2 to fail after a number of failed > attempts to obtain partition ownership. > In other words, AbstractFetcherThread.shutdown() is not guaranteed to > shutdown due to this bug. > The problem is confirmed with Java 1.7 and java 1.6. To check it by yourself, > you can use the attached server.java and client.java -- start the server > before the client and see if client unblock after interruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)
[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584140#comment-14584140 ] Jun Rao commented on KAFKA-2241: [~lindong], thanks for reporting this. Does this happen on java 8? > AbstractFetcherThread.shutdown() should not block on > ReadableByteChannel.read(buffer) > - > > Key: KAFKA-2241 > URL: https://issues.apache.org/jira/browse/KAFKA-2241 > Project: Kafka > Issue Type: Bug >Reporter: Dong Lin >Assignee: Dong Lin > Labels: quotas > Attachments: KAFKA-2241.patch, KAFKA-2241_2015-06-03_15:30:35.patch, > client.java, server.java > > > This is likely a bug from Java. This affects Kafka and here is the patch to > fix it. > Here is the description of the bug. By description of SocketChannel in Java 7 > Documentation. If another thread interrupts the current thread while the read > operation is in progress, the it should closes the channel and throw > ClosedByInterruptException. However, we find that interrupting the thread > will not unblock the channel immediately. Instead, it waits for response or > socket timeout before throwing an exception. > This will cause problem in the following scenario. Suppose one > console_consumer_1 is reading from a topic, and due to quota delay or > whatever reason, it block on channel.read(buffer). At this moment, another > console_consumer_2 joins and triggers rebalance at console_consumer_1. But > consumer_1 will block waiting on the channel.read before it can release > partition ownership, causing consumer_2 to fail after a number of failed > attempts to obtain partition ownership. > In other words, AbstractFetcherThread.shutdown() is not guaranteed to > shutdown due to this bug. > The problem is confirmed with Java 1.7 and java 1.6. To check it by yourself, > you can use the attached server.java and client.java -- start the server > before the client and see if client unblock after interruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)
[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571766#comment-14571766 ] Dong Lin commented on KAFKA-2241: - Updated reviewboard https://reviews.apache.org/r/34965/diff/ against branch origin/trunk > AbstractFetcherThread.shutdown() should not block on > ReadableByteChannel.read(buffer) > - > > Key: KAFKA-2241 > URL: https://issues.apache.org/jira/browse/KAFKA-2241 > Project: Kafka > Issue Type: Bug >Reporter: Dong Lin >Assignee: Dong Lin > Attachments: KAFKA-2241.patch, KAFKA-2241_2015-06-03_15:30:35.patch, > client.java, server.java > > > This is likely a bug from Java. This affects Kafka and here is the patch to > fix it. > Here is the description of the bug. By description of SocketChannel in Java 7 > Documentation. If another thread interrupts the current thread while the read > operation is in progress, the it should closes the channel and throw > ClosedByInterruptException. However, we find that interrupting the thread > will not unblock the channel immediately. Instead, it waits for response or > socket timeout before throwing an exception. > This will cause problem in the following scenario. Suppose one > console_consumer_1 is reading from a topic, and due to quota delay or > whatever reason, it block on channel.read(buffer). At this moment, another > console_consumer_2 joins and triggers rebalance at console_consumer_1. But > consumer_1 will block waiting on the channel.read before it can release > partition ownership, causing consumer_2 to fail after a number of failed > attempts to obtain partition ownership. > In other words, AbstractFetcherThread.shutdown() is not guaranteed to > shutdown due to this bug. > The problem is confirmed with Java 1.7 and java 1.6. To check it by yourself, > you can use the attached server.java and client.java -- start the server > before the client and see if client unblock after interruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)
[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14570010#comment-14570010 ] Dong Lin commented on KAFKA-2241: - Created reviewboard https://reviews.apache.org/r/34965/diff/ against branch origin/trunk > AbstractFetcherThread.shutdown() should not block on > ReadableByteChannel.read(buffer) > - > > Key: KAFKA-2241 > URL: https://issues.apache.org/jira/browse/KAFKA-2241 > Project: Kafka > Issue Type: Bug >Reporter: Dong Lin >Assignee: Dong Lin > Attachments: KAFKA-2241.patch, client.java, server.java > > > This is likely a bug from Java. This affects Kafka and here is the patch to > fix it. > Here is the description of the bug. By description of SocketChannel in Java 7 > Documentation. If another thread interrupts the current thread while the read > operation is in progress, the it should closes the channel and throw > ClosedByInterruptException. However, we find that interrupting the thread > will not unblock the channel immediately. Instead, it waits for response or > socket timeout before throwing an exception. > This will cause problem in the following scenario. Suppose one > console_consumer_1 is reading from a topic, and due to quota delay or > whatever reason, it block on channel.read(buffer). At this moment, another > console_consumer_2 joins and triggers rebalance at console_consumer_1. But > consumer_1 will block waiting on the channel.read before it can release > partition ownership, causing consumer_2 to fail after a number of failed > attempts to obtain partition ownership. > In other words, AbstractFetcherThread.shutdown() is not guaranteed to > shutdown due to this bug. > The problem is confirmed with Java 1.7 and java 1.6. To check it by yourself, > you can use the attached server.java and client.java -- start the server > before the client and see if client unblock after interruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)
[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569988#comment-14569988 ] Dong Lin commented on KAFKA-2241: - Oh, I thought the first issue creation has failed. Thanks for your notice. > AbstractFetcherThread.shutdown() should not block on > ReadableByteChannel.read(buffer) > - > > Key: KAFKA-2241 > URL: https://issues.apache.org/jira/browse/KAFKA-2241 > Project: Kafka > Issue Type: Bug >Reporter: Dong Lin >Assignee: Dong Lin > Attachments: client.java, server.java > > > This is likely a bug from Java. This affects Kafka and here is the patch to > fix it. > Here is the description of the bug. By description of SocketChannel in Java 7 > Documentation. If another thread interrupts the current thread while the read > operation is in progress, the it should closes the channel and throw > ClosedByInterruptException. However, we find that interrupting the thread > will not unblock the channel immediately. Instead, it waits for response or > socket timeout before throwing an exception. > This will cause problem in the following scenario. Suppose one > console_consumer_1 is reading from a topic, and due to quota delay or > whatever reason, it block on channel.read(buffer). At this moment, another > console_consumer_2 joins and triggers rebalance at console_consumer_1. But > consumer_1 will block waiting on the channel.read before it can release > partition ownership, causing consumer_2 to fail after a number of failed > attempts to obtain partition ownership. > In other words, AbstractFetcherThread.shutdown() is not guaranteed to > shutdown due to this bug. > The problem is confirmed with Java 1.7 and java 1.6. To check it by yourself, > you can use the attached server.java and client.java -- start the server > before the client and see if client unblock after interruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2241) AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer)
[ https://issues.apache.org/jira/browse/KAFKA-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569934#comment-14569934 ] Ismael Juma commented on KAFKA-2241: [~lindong], it looks like you filed two issues for the same thing KAFKA-2240 and KAFKA-2241. Maybe close one of them as a duplicate? > AbstractFetcherThread.shutdown() should not block on > ReadableByteChannel.read(buffer) > - > > Key: KAFKA-2241 > URL: https://issues.apache.org/jira/browse/KAFKA-2241 > Project: Kafka > Issue Type: Bug >Reporter: Dong Lin >Assignee: Dong Lin > Attachments: client.java, server.java > > > This is likely a bug from Java. This affects Kafka and here is the patch to > fix it. > Here is the description of the bug. By description of SocketChannel in Java 7 > Documentation. If another thread interrupts the current thread while the read > operation is in progress, the it should closes the channel and throw > ClosedByInterruptException. However, we find that interrupting the thread > will not unblock the channel immediately. Instead, it waits for response or > socket timeout before throwing an exception. > This will cause problem in the following scenario. Suppose one > console_consumer_1 is reading from a topic, and due to quota delay or > whatever reason, it block on channel.read(buffer). At this moment, another > console_consumer_2 joins and triggers rebalance at console_consumer_1. But > consumer_1 will block waiting on the channel.read before it can release > partition ownership, causing consumer_2 to fail after a number of failed > attempts to obtain partition ownership. > In other words, AbstractFetcherThread.shutdown() is not guaranteed to > shutdown due to this bug. > The problem is confirmed with Java 1.7 and java 1.6. To check it by yourself, > you can use the attached server.java and client.java -- start the server > before the client and see if client unblock after interruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
