[jira] [Commented] (KAFKA-1972) JMX Tool output for CSV format does not handle attributes with comma in their value
[ https://issues.apache.org/jira/browse/KAFKA-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14332387#comment-14332387 ] Rekha Joshi commented on KAFKA-1972: [~jrafalski] Patch at https://github.com/apache/kafka/pull/45 Thanks! > JMX Tool output for CSV format does not handle attributes with comma in their > value > --- > > Key: KAFKA-1972 > URL: https://issues.apache.org/jira/browse/KAFKA-1972 > Project: Kafka > Issue Type: Bug > Components: tools >Affects Versions: 0.8.1.1 >Reporter: Jonathan Rafalski >Priority: Minor > Labels: newbie > > When the JMXTools outputs all attributes using a comma delimitation it does > not have an exit character or a way to handle attributes that contain comma's > in their value. This could potentially limit the uses of the output to > single value attributes only. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1621) Standardize --messages option in perf scripts
[ https://issues.apache.org/jira/browse/KAFKA-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-1621: --- Reviewer: Jay Kreps Status: Patch Available (was: Open) https://github.com/apache/kafka/pull/46 > Standardize --messages option in perf scripts > - > > Key: KAFKA-1621 > URL: https://issues.apache.org/jira/browse/KAFKA-1621 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.1.1 >Reporter: Jay Kreps > Labels: newbie > > This option is specified in PerfConfig and is used by the producer, consumer > and simple consumer perf commands. The docstring on the argument does not > list it as required but the producer performance test requires it--others > don't. > We should standardize this so that either all the commands require the option > and it is marked as required in the docstring or none of them list it as > required. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1972) JMX Tool output for CSV format does not handle attributes with comma in their value
[ https://issues.apache.org/jira/browse/KAFKA-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-1972: --- Reviewer: Jonathan Rafalski Status: Patch Available (was: Open) Patch at https://github.com/apache/kafka/pull/45 Thanks! > JMX Tool output for CSV format does not handle attributes with comma in their > value > --- > > Key: KAFKA-1972 > URL: https://issues.apache.org/jira/browse/KAFKA-1972 > Project: Kafka > Issue Type: Bug > Components: tools >Affects Versions: 0.8.1.1 >Reporter: Jonathan Rafalski >Priority: Minor > Labels: newbie > > When the JMXTools outputs all attributes using a comma delimitation it does > not have an exit character or a way to handle attributes that contain comma's > in their value. This could potentially limit the uses of the output to > single value attributes only. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1545) java.net.InetAddress.getLocalHost in KafkaHealthcheck.register may fail on some irregular hostnames
[ https://issues.apache.org/jira/browse/KAFKA-1545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-1545: --- Reviewer: Guozhang Wang Affects Version/s: 0.8.1.1 Status: Patch Available (was: Open) > java.net.InetAddress.getLocalHost in KafkaHealthcheck.register may fail on > some irregular hostnames > --- > > Key: KAFKA-1545 > URL: https://issues.apache.org/jira/browse/KAFKA-1545 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.1.1 >Reporter: Guozhang Wang >Assignee: Guozhang Wang > Labels: newbie > Fix For: 0.9.0 > > > For example: > kafka.server.LogOffsetTest > testGetOffsetsForUnknownTopic FAILED > java.net.UnknownHostException: guwang-mn2: guwang-mn2: nodename nor > servname provided, or not known > at java.net.InetAddress.getLocalHost(InetAddress.java:1473) > at kafka.server.KafkaHealthcheck.register(KafkaHealthcheck.scala:59) > at kafka.server.KafkaHealthcheck.startup(KafkaHealthcheck.scala:45) > at kafka.server.KafkaServer.startup(KafkaServer.scala:121) > at kafka.utils.TestUtils$.createServer(TestUtils.scala:130) > at kafka.server.LogOffsetTest.setUp(LogOffsetTest.scala:53) > Caused by: > java.net.UnknownHostException: guwang-mn2: nodename nor servname > provided, or not known > at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) > at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901) > at > java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293) > at java.net.InetAddress.getLocalHost(InetAddress.java:1469) > ... 5 more -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-269) ./system_test/producer_perf/bin/run-test.sh without --async flag does not run
[ https://issues.apache.org/jira/browse/KAFKA-269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-269: -- Reviewer: Jay Kreps Affects Version/s: 0.8.1.1 Status: Patch Available (was: Open) [~praveen27] By default if no --sync option is provided the producer run is in async mode. Ran the run-test.sh without --async option and works on 0.8.2.or maybe you can check your zk/topic/reporting-interval? start producing 200 messages ... start.time, end.time, compression, message.size, batch.size, total.data.sent.in.MB, MB.sec, total.data.sent.in.nMsg, nMsg.sec SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/Users/rjoshi2/Documents/code/kafka-fork/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/Users/rjoshi2/Documents/code/kafka-fork/kafka/core/build/dependant-libs-2.10.4/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 2015-02-22 17:35:18:471, 2015-02-22 17:35:24:331, 0, 200, 200, 381.47, 65.0972, 200, 341296.9283 wait for data to be persisted The patch as, AFAIU, 'async' is not a recognized option.Only - -sync is. Thanks. > ./system_test/producer_perf/bin/run-test.sh without --async flag does not run > -- > > Key: KAFKA-269 > URL: https://issues.apache.org/jira/browse/KAFKA-269 > Project: Kafka > Issue Type: Bug > Components: clients, core >Affects Versions: 0.8.1.1, 0.7 > Environment: Linux 2.6.18-238.1.1.el5 , x86_64 x86_64 x86_64 GNU/Linux > ext3 file system with raid10 >Reporter: Praveen Ramachandra > Labels: newbie, performance > > When I run the tests without --async option, The tests doesn't produce even a > single message. > Following defaults where changed in the server.properties > num.threads=Tried with 8, 10, 100 > num.partitions=10 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-724) Allow automatic socket.send.buffer from operating system
[ https://issues.apache.org/jira/browse/KAFKA-724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-724: -- Reviewer: Jay Kreps Affects Version/s: 0.8.2.0 Status: Patch Available (was: Open) > Allow automatic socket.send.buffer from operating system > > > Key: KAFKA-724 > URL: https://issues.apache.org/jira/browse/KAFKA-724 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.8.2.0 >Reporter: Pablo Barrera > Labels: newbie > > To do this, don't call to socket().setXXXBufferSize. This can be > controlled by the configuration parameter: if the value socket.send.buffer or > others are set to -1, don't call to socket().setXXXBufferSize -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-1995) JMS to Kafka: Inbuilt JMSAdaptor/JMSProxy/JMSBridge (Client can speak JMS but hit Kafka)
Rekha Joshi created KAFKA-1995: -- Summary: JMS to Kafka: Inbuilt JMSAdaptor/JMSProxy/JMSBridge (Client can speak JMS but hit Kafka) Key: KAFKA-1995 URL: https://issues.apache.org/jira/browse/KAFKA-1995 Project: Kafka Issue Type: Wish Components: core Affects Versions: 0.8.3 Reporter: Rekha Joshi Kafka is a great alternative to JMS, providing high performance, throughput as scalable, distributed pub sub/commit log service. However there always exist traditional systems running on JMS. Rather than rewriting, it would be great if we just had an inbuilt JMSAdaptor/JMSProxy/JMSBridge by which client can speak JMS but hit Kafka behind-the-scene. Something like Chukwa's o.a.h.chukwa.datacollection.adaptor.jms.JMSAdaptor, which receives msg off JMS queue and transforms to a Chukwa chunk? I have come across folks talking of this need in past as well.Is it considered and/or part of the roadmap? http://grokbase.com/t/kafka/users/131cst8xpv/stomp-binding-for-kafka http://grokbase.com/t/kafka/users/148dm4247q/consuming-messages-from-kafka-and-pushing-on-to-a-jms-queue http://grokbase.com/t/kafka/users/143hjepbn2/request-kafka-zookeeper-jms-details Looking for inputs on correct way to approach this so to retain all good features of Kafka while still not rewriting entire application.Possible? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1995) JMS to Kafka: Inbuilt JMSAdaptor/JMSProxy/JMSBridge (Client can speak JMS but hit Kafka)
[ https://issues.apache.org/jira/browse/KAFKA-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342819#comment-14342819 ] Rekha Joshi commented on KAFKA-1995: [~jkreps] [~nehanarkhede] [~junrao] [~joestein] it would be great to know your thoughts on it.thanks. > JMS to Kafka: Inbuilt JMSAdaptor/JMSProxy/JMSBridge (Client can speak JMS but > hit Kafka) > > > Key: KAFKA-1995 > URL: https://issues.apache.org/jira/browse/KAFKA-1995 > Project: Kafka > Issue Type: Wish > Components: core >Affects Versions: 0.8.3 >Reporter: Rekha Joshi > > Kafka is a great alternative to JMS, providing high performance, throughput > as scalable, distributed pub sub/commit log service. > However there always exist traditional systems running on JMS. > Rather than rewriting, it would be great if we just had an inbuilt > JMSAdaptor/JMSProxy/JMSBridge by which client can speak JMS but hit Kafka > behind-the-scene. > Something like Chukwa's o.a.h.chukwa.datacollection.adaptor.jms.JMSAdaptor, > which receives msg off JMS queue and transforms to a Chukwa chunk? > I have come across folks talking of this need in past as well.Is it > considered and/or part of the roadmap? > http://grokbase.com/t/kafka/users/131cst8xpv/stomp-binding-for-kafka > http://grokbase.com/t/kafka/users/148dm4247q/consuming-messages-from-kafka-and-pushing-on-to-a-jms-queue > http://grokbase.com/t/kafka/users/143hjepbn2/request-kafka-zookeeper-jms-details > Looking for inputs on correct way to approach this so to retain all good > features of Kafka while still not rewriting entire application.Possible? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1995) JMS to Kafka: Inbuilt JMSAdaptor/JMSProxy/JMSBridge (Client can speak JMS but hit Kafka)
[ https://issues.apache.org/jira/browse/KAFKA-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349493#comment-14349493 ] Rekha Joshi commented on KAFKA-1995: Great.Thanks for your reply, [~ewencp] [~joestein] ! > JMS to Kafka: Inbuilt JMSAdaptor/JMSProxy/JMSBridge (Client can speak JMS but > hit Kafka) > > > Key: KAFKA-1995 > URL: https://issues.apache.org/jira/browse/KAFKA-1995 > Project: Kafka > Issue Type: Wish > Components: core >Affects Versions: 0.8.3 >Reporter: Rekha Joshi > > Kafka is a great alternative to JMS, providing high performance, throughput > as scalable, distributed pub sub/commit log service. > However there always exist traditional systems running on JMS. > Rather than rewriting, it would be great if we just had an inbuilt > JMSAdaptor/JMSProxy/JMSBridge by which client can speak JMS but hit Kafka > behind-the-scene. > Something like Chukwa's o.a.h.chukwa.datacollection.adaptor.jms.JMSAdaptor, > which receives msg off JMS queue and transforms to a Chukwa chunk? > I have come across folks talking of this need in past as well.Is it > considered and/or part of the roadmap? > http://grokbase.com/t/kafka/users/131cst8xpv/stomp-binding-for-kafka > http://grokbase.com/t/kafka/users/148dm4247q/consuming-messages-from-kafka-and-pushing-on-to-a-jms-queue > http://grokbase.com/t/kafka/users/143hjepbn2/request-kafka-zookeeper-jms-details > Looking for inputs on correct way to approach this so to retain all good > features of Kafka while still not rewriting entire application.Possible? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1621) Standardize --messages option in perf scripts
[ https://issues.apache.org/jira/browse/KAFKA-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14481865#comment-14481865 ] Rekha Joshi commented on KAFKA-1621: Hi.missed updating earlier. everything else was setup as https://cwiki.apache.org/confluence/display/KAFKA/Patch+submission+and+review but the jira-client could not be installed (Mac 10.9.5, Python 2.7.5, pip 1.5.6), hence could not update review board. suggestion: why not review directly on git as rest of the apache projects? python kafka-patch-review.py --help Traceback (most recent call last): File "kafka-patch-review.py", line 10, in from jira.client import JIRA ImportError: No module named jira.client pip install jira-python Downloading/unpacking jira-python Could not find any downloads that satisfy the requirement jira-python Cleaning up... No distributions at all found for jira-python Storing debug log for failure in /usr/local/.pip/pip.log sudo easy_install jira-python Searching for jira-python Reading http://pypi.python.org/simple/jira-python/ Couldn't find index page for 'jira-python' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://pypi.python.org/simple/ No local packages or download links found for jira-python error: Could not find suitable distribution for Requirement.parse('jira-python') Think it could be related to security enabled on pip?, but I would prefer not to downgrade pip as used for my other projects as well. > Standardize --messages option in perf scripts > - > > Key: KAFKA-1621 > URL: https://issues.apache.org/jira/browse/KAFKA-1621 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.1.1 >Reporter: Jay Kreps > Labels: newbie > > This option is specified in PerfConfig and is used by the producer, consumer > and simple consumer perf commands. The docstring on the argument does not > list it as required but the producer performance test requires it--others > don't. > We should standardize this so that either all the commands require the option > and it is marked as required in the docstring or none of them list it as > required. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-2098) Gradle Wrapper Jar gone missing in 0.8.2.1
Rekha Joshi created KAFKA-2098: -- Summary: Gradle Wrapper Jar gone missing in 0.8.2.1 Key: KAFKA-2098 URL: https://issues.apache.org/jira/browse/KAFKA-2098 Project: Kafka Issue Type: Bug Components: build Affects Versions: 0.8.2.1 Reporter: Rekha Joshi ./gradlew idea Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain This was working in 0.8.2.Attaching patch.Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-2098) Gradle Wrapper Jar gone missing in 0.8.2.1
[ https://issues.apache.org/jira/browse/KAFKA-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi reassigned KAFKA-2098: -- Assignee: Rekha Joshi > Gradle Wrapper Jar gone missing in 0.8.2.1 > -- > > Key: KAFKA-2098 > URL: https://issues.apache.org/jira/browse/KAFKA-2098 > Project: Kafka > Issue Type: Bug > Components: build >Affects Versions: 0.8.2.1 >Reporter: Rekha Joshi >Assignee: Rekha Joshi > > ./gradlew idea > Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain > This was working in 0.8.2.Attaching patch.Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2098) Gradle Wrapper Jar gone missing in 0.8.2.1
[ https://issues.apache.org/jira/browse/KAFKA-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497350#comment-14497350 ] Rekha Joshi commented on KAFKA-2098: Hi.I agree [~alexcb] people all over the world are losing time on it :), though I also understand where [~gwenshap] is coming from. For users point of view better to have it in, especially as many standard open source based on gradle have them, one that comes to mind immediately is Samza - https://github.com/apache/samza/tree/master/gradle/wrapper Anyhow I will let Kafka committers and [~jkreps] decide whats works best. Thanks Rekha > Gradle Wrapper Jar gone missing in 0.8.2.1 > -- > > Key: KAFKA-2098 > URL: https://issues.apache.org/jira/browse/KAFKA-2098 > Project: Kafka > Issue Type: Bug > Components: build >Affects Versions: 0.8.2.1 >Reporter: Rekha Joshi >Assignee: Rekha Joshi > > ./gradlew idea > Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain > This was working in 0.8.2.Attaching patch.Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2098) Gradle Wrapper Jar gone missing in 0.8.2.1
[ https://issues.apache.org/jira/browse/KAFKA-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497644#comment-14497644 ] Rekha Joshi commented on KAFKA-2098: Thanks for a prompt reply [~jkreps].What works for you works for me :-) But just for my curiosity, can you please share link to this Apache requirement? As Samza is also a top level Apache project and has https://github.com/apache/samza/tree/master/gradle/wrapper Thanks! > Gradle Wrapper Jar gone missing in 0.8.2.1 > -- > > Key: KAFKA-2098 > URL: https://issues.apache.org/jira/browse/KAFKA-2098 > Project: Kafka > Issue Type: Bug > Components: build >Affects Versions: 0.8.2.1 >Reporter: Rekha Joshi >Assignee: Rekha Joshi > > ./gradlew idea > Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain > This was working in 0.8.2.Attaching patch.Thanks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1621) Standardize --messages option in perf scripts
[ https://issues.apache.org/jira/browse/KAFKA-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14515003#comment-14515003 ] Rekha Joshi commented on KAFKA-1621: No worries [~nehanarkhede] Thanks for your review, PerfConfig was found updated in trunk for REQUIRED, rest applied on pull #58.Thanks. > Standardize --messages option in perf scripts > - > > Key: KAFKA-1621 > URL: https://issues.apache.org/jira/browse/KAFKA-1621 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.1.1 >Reporter: Jay Kreps > Labels: newbie > > This option is specified in PerfConfig and is used by the producer, consumer > and simple consumer perf commands. The docstring on the argument does not > list it as required but the producer performance test requires it--others > don't. > We should standardize this so that either all the commands require the option > and it is marked as required in the docstring or none of them list it as > required. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-1284) Received -1 when reading from channel, socket has likely been closed.
[ https://issues.apache.org/jira/browse/KAFKA-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi reassigned KAFKA-1284: -- Assignee: Rekha Joshi > Received -1 when reading from channel, socket has likely been closed. > - > > Key: KAFKA-1284 > URL: https://issues.apache.org/jira/browse/KAFKA-1284 > Project: Kafka > Issue Type: Bug >Reporter: darion yaphets >Assignee: Rekha Joshi >Priority: Trivial > > I use kafka 0.7.2 as storm data source (0.8.1) and storm-kafka maybe a > useful spout to feed data into storm > but storm report Received -1 when reading from channel, socket has likely > been closed. at storm spout in open() . It's confuse and any one could help > me . Thanks a lot ~~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (KAFKA-1284) Received -1 when reading from channel, socket has likely been closed.
[ https://issues.apache.org/jira/browse/KAFKA-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-1284 started by Rekha Joshi. -- > Received -1 when reading from channel, socket has likely been closed. > - > > Key: KAFKA-1284 > URL: https://issues.apache.org/jira/browse/KAFKA-1284 > Project: Kafka > Issue Type: Bug >Reporter: darion yaphets >Assignee: Rekha Joshi >Priority: Trivial > > I use kafka 0.7.2 as storm data source (0.8.1) and storm-kafka maybe a > useful spout to feed data into storm > but storm report Received -1 when reading from channel, socket has likely > been closed. at storm spout in open() . It's confuse and any one could help > me . Thanks a lot ~~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1284) Received -1 when reading from channel, socket has likely been closed.
[ https://issues.apache.org/jira/browse/KAFKA-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14515069#comment-14515069 ] Rekha Joshi commented on KAFKA-1284: Hi [~darion] The -1 error will be thrown only if the channel is closed, has reached end-of-stream - http://docs.oracle.com/javase/7/docs/api/java/nio/channels/ReadableByteChannel.html Please check your configurations are correct, nimbus, zk are good at storm end, and there are no firewall issues? I use later versions of kafka storm for real time data ingestion pipeline and they work fine.If you could provide complete stack trace post confirming all the configs/ps are good? Maybe it could be older versions issue.Also it might help you to check https://github.com/miguno/kafka-storm-starter Thanks. > Received -1 when reading from channel, socket has likely been closed. > - > > Key: KAFKA-1284 > URL: https://issues.apache.org/jira/browse/KAFKA-1284 > Project: Kafka > Issue Type: Bug >Reporter: darion yaphets >Assignee: Rekha Joshi >Priority: Trivial > > I use kafka 0.7.2 as storm data source (0.8.1) and storm-kafka maybe a > useful spout to feed data into storm > but storm report Received -1 when reading from channel, socket has likely > been closed. at storm spout in open() . It's confuse and any one could help > me . Thanks a lot ~~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KAFKA-1284) Received -1 when reading from channel, socket has likely been closed.
[ https://issues.apache.org/jira/browse/KAFKA-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi resolved KAFKA-1284. Resolution: Cannot Reproduce > Received -1 when reading from channel, socket has likely been closed. > - > > Key: KAFKA-1284 > URL: https://issues.apache.org/jira/browse/KAFKA-1284 > Project: Kafka > Issue Type: Bug >Reporter: darion yaphets >Assignee: Rekha Joshi >Priority: Trivial > > I use kafka 0.7.2 as storm data source (0.8.1) and storm-kafka maybe a > useful spout to feed data into storm > but storm report Received -1 when reading from channel, socket has likely > been closed. at storm spout in open() . It's confuse and any one could help > me . Thanks a lot ~~ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-1508) Scripts Break When Path Has Spaces
[ https://issues.apache.org/jira/browse/KAFKA-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi reassigned KAFKA-1508: -- Assignee: Rekha Joshi > Scripts Break When Path Has Spaces > -- > > Key: KAFKA-1508 > URL: https://issues.apache.org/jira/browse/KAFKA-1508 > Project: Kafka > Issue Type: Bug > Components: tools >Affects Versions: 0.8.1.1 > Environment: Any *nix flavor where the full path name to the Kafka > deployment contains spaces >Reporter: Tim Olson >Assignee: Rekha Joshi >Priority: Minor > > All the shell scripts in {{bin}} use the idom > {{$(dirname $0)}} > but this produces the error > {{usage: dirname path}} > if the path contains spaces. The correct way to get the dirname is to use: > {{"$(dirname "$0")"}} > and subsequently wrap the result in quotes when it is used. For example, the > file {{bin/kafka-run-class.sh}} should look like this starting line 23: > {code} > # BUGFIX: quotes added > base_dir="$(dirname "$0")/.." > # create logs directory > # BUGFIX: quotes added > LOG_DIR="$base_dir/logs" > if [ ! -d "$LOG_DIR" ]; then > # BUGFIX: quotes added around $LOG_DIR > mkdir "$LOG_DIR" > fi > # ... > # BUGFIX: quotes added > for file in "$base_dir/core/build/dependant-libs-${SCALA_VERSION}/*.jar"; > do > CLASSPATH=$CLASSPATH:"$file" > done > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (KAFKA-3893) Kafka Borker ID disappears from /borkers/ids
[ https://issues.apache.org/jira/browse/KAFKA-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616914#comment-15616914 ] Rekha Joshi edited comment on KAFKA-3893 at 11/30/16 2:58 AM: -- We saw it too in Kafka 0.8.x and upgraded to Kafka 0.9.x and continued seeing this issue, especially in AWS.This was seen in AWS especially if you configure zookeeper connect property with zookeeper ELB instead of individual IP:port set. When we reversed zookeeper connect with zookeeper IP:port instead of zookeeper ELB, it worked great. It would be great(can help) if Apache Kafka/Confluent FAQ can be updated to note this. Thanks Rekha was (Author: rekhajoshm): We saw it too in Kafka 0.8.x and upgraded to Kafka 0.9.x and continued seeing this issue.Going with workarounds.But it would be great(can help) if this is documented clearly on Apache Kafka/Confluent FAQ. Thanks Rekha > Kafka Borker ID disappears from /borkers/ids > > > Key: KAFKA-3893 > URL: https://issues.apache.org/jira/browse/KAFKA-3893 > Project: Kafka > Issue Type: Bug >Reporter: chaitra >Priority: Critical > > Kafka version used : 0.8.2.1 > Zookeeper version: 3.4.6 > We have scenario where kafka 's broker in zookeeper path /brokers/ids just > disappears. > We see the zookeeper connection active and no network issue. > The zookeeper conection timeout is set to 6000ms in server.properties > Hence Kafka not participating in cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3893) Kafka Broker ID disappears from /borkers/ids
[ https://issues.apache.org/jira/browse/KAFKA-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-3893: --- Summary: Kafka Broker ID disappears from /borkers/ids (was: Kafka Borker ID disappears from /borkers/ids) > Kafka Broker ID disappears from /borkers/ids > > > Key: KAFKA-3893 > URL: https://issues.apache.org/jira/browse/KAFKA-3893 > Project: Kafka > Issue Type: Bug >Reporter: chaitra >Priority: Critical > > Kafka version used : 0.8.2.1 > Zookeeper version: 3.4.6 > We have scenario where kafka 's broker in zookeeper path /brokers/ids just > disappears. > We see the zookeeper connection active and no network issue. > The zookeeper conection timeout is set to 6000ms in server.properties > Hence Kafka not participating in cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3893) Kafka Broker ID disappears from /brokers/ids
[ https://issues.apache.org/jira/browse/KAFKA-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-3893: --- Summary: Kafka Broker ID disappears from /brokers/ids (was: Kafka Broker ID disappears from /borkers/ids) > Kafka Broker ID disappears from /brokers/ids > > > Key: KAFKA-3893 > URL: https://issues.apache.org/jira/browse/KAFKA-3893 > Project: Kafka > Issue Type: Bug >Reporter: chaitra >Priority: Critical > > Kafka version used : 0.8.2.1 > Zookeeper version: 3.4.6 > We have scenario where kafka 's broker in zookeeper path /brokers/ids just > disappears. > We see the zookeeper connection active and no network issue. > The zookeeper conection timeout is set to 6000ms in server.properties > Hence Kafka not participating in cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-4500) Code Corrections
Rekha Joshi created KAFKA-4500: -- Summary: Code Corrections Key: KAFKA-4500 URL: https://issues.apache.org/jira/browse/KAFKA-4500 Project: Kafka Issue Type: Improvement Components: clients Affects Versions: 0.9.0.2 Reporter: Rekha Joshi Assignee: Rekha Joshi Priority: Trivial Code Corrections on clients module -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3796) SslTransportLayerTest.testInvalidEndpointIdentification fails on trunk
[ https://issues.apache.org/jira/browse/KAFKA-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-3796: --- Priority: Trivial (was: Major) Issue Type: Improvement (was: Bug) > SslTransportLayerTest.testInvalidEndpointIdentification fails on trunk > -- > > Key: KAFKA-3796 > URL: https://issues.apache.org/jira/browse/KAFKA-3796 > Project: Kafka > Issue Type: Improvement > Components: clients, security >Affects Versions: 0.10.0.0 >Reporter: Rekha Joshi >Assignee: Rekha Joshi >Priority: Trivial > > org.apache.kafka.common.network.SslTransportLayerTest > > testEndpointIdentificationDisabled FAILED > java.net.BindException: Can't assign requested address > at sun.nio.ch.Net.bind0(Native Method) > at sun.nio.ch.Net.bind(Net.java:433) > at sun.nio.ch.Net.bind(Net.java:425) > at > sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67) > at > org.apache.kafka.common.network.NioEchoServer.(NioEchoServer.java:48) > at > org.apache.kafka.common.network.SslTransportLayerTest.testEndpointIdentificationDisabled(SslTransportLayerTest.java:120) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-4500) Code Improvements
[ https://issues.apache.org/jira/browse/KAFKA-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-4500: --- Summary: Code Improvements (was: Code Corrections) > Code Improvements > - > > Key: KAFKA-4500 > URL: https://issues.apache.org/jira/browse/KAFKA-4500 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.9.0.2 >Reporter: Rekha Joshi >Assignee: Rekha Joshi >Priority: Trivial > > Code Corrections on clients module -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-4500) Code Improvements
[ https://issues.apache.org/jira/browse/KAFKA-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-4500: --- Priority: Minor (was: Trivial) > Code Improvements > - > > Key: KAFKA-4500 > URL: https://issues.apache.org/jira/browse/KAFKA-4500 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.9.0.2 >Reporter: Rekha Joshi >Assignee: Rekha Joshi >Priority: Minor > > Code Corrections on clients module -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-4500) Kafka Code Improvements
[ https://issues.apache.org/jira/browse/KAFKA-4500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-4500: --- Summary: Kafka Code Improvements (was: Code Improvements) > Kafka Code Improvements > --- > > Key: KAFKA-4500 > URL: https://issues.apache.org/jira/browse/KAFKA-4500 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.9.0.2 >Reporter: Rekha Joshi >Assignee: Rekha Joshi >Priority: Minor > > Code Corrections on clients module -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-4555) Using Hamcrest for easy intent expression in tests
Rekha Joshi created KAFKA-4555: -- Summary: Using Hamcrest for easy intent expression in tests Key: KAFKA-4555 URL: https://issues.apache.org/jira/browse/KAFKA-4555 Project: Kafka Issue Type: Test Components: unit tests Affects Versions: 0.9.0.1 Reporter: Rekha Joshi Assignee: Rekha Joshi Priority: Minor Using Hamcrest for easy intent expression in tests -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3893) Kafka Borker ID disappears from /borkers/ids
[ https://issues.apache.org/jira/browse/KAFKA-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616914#comment-15616914 ] Rekha Joshi commented on KAFKA-3893: We saw it too in Kafka 0.8.x and upgraded to Kafka 0.9.x and continued seeing this issue.Going with workarounds.But it would be great(can help) if this is documented clearly on Apache Kafka/Confluent FAQ. Thanks Rekha > Kafka Borker ID disappears from /borkers/ids > > > Key: KAFKA-3893 > URL: https://issues.apache.org/jira/browse/KAFKA-3893 > Project: Kafka > Issue Type: Bug >Reporter: chaitra >Priority: Critical > > Kafka version used : 0.8.2.1 > Zookeeper version: 3.4.6 > We have scenario where kafka 's broker in zookeeper path /brokers/ids just > disappears. > We see the zookeeper connection active and no network issue. > The zookeeper conection timeout is set to 6000ms in server.properties > Hence Kafka not participating in cluster -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-914) Deadlock between initial rebalance and watcher-triggered rebalances
[ https://issues.apache.org/jira/browse/KAFKA-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145519#comment-15145519 ] Rekha Joshi commented on KAFKA-914: --- Hi, We have been seeing consistent issue mirroring between our DataCenters., and same issue seems to resurface. Below is the setup details Source: AWS (13 Brokers) Destination: OTHER-DC (20 Brokers) Topic: mirrortest (source: 260 partitions, destination: 200 partitions) Connectivity: AWS Direct Connect (max 6Gbps) Data details: Source is receiving 40,000 msg/sec, each message is around 5KB Mirroring Node: is at our OTHER-DC (24 Cores, 48 GB Memory) JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 Launch script: kafka.tools.MirrorMaker --consumer.config consumer.properties --producer.config producer.properties --num.producers 1 --whitelist mirrortest --num.streams 1 --queue.size 10 consumer.properties --- zookeeper.connect= group.id=KafkaMirror auto.offset.reset=smallest fetch.message.max.bytes=900 zookeeper.connection.timeout.ms=6 rebalance.max.retries=4 rebalance.backoff.ms=5000 producer.properties -- metadata.broker.list= partitioner.class= producer.type=async When we start the mirroring job everything works fine as expected, Eventually we hit an issue where the job stops consuming no more. At this stage: {code} 1. No Error seen in the mirrormaker logs 2. consumer threads are not fetching any messages and we see thread dumps as follows: "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread t@73 java.lang.Thread.State: WAITING at sun.misc.Unsafe.park(Native Method) - parking to wait for <79b6d3ce> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa i t(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 ) at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) at kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher T hread.scala:49) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu n $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu n $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m c V$sp(AbstractFetcherThread.scala:109) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A b stractFetcherThread.scala:109) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A b stractFetcherThread.scala:109) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr e ad.scala:108) at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) Locked ownable synchronizers: - locked <199dc92d> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) 3. Producer stops producing, in trace mode we notice it's handling 0 events and Thread dump as follows: "ProducerSendThread--0" - Thread t@53 java.lang.Thread.State: RUNNABLE at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.write(IOUtil.java:148) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) - locked <5ae2fc40> (a java.lang.Object) at java.nio.channels.SocketChannel.write(SocketChannel.java:502) at kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:5 6 ) at kafka.network.Send$class.writeCompletely(Transmission.scala:75) at kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend . scala:26) at kafka.network.BlockingChannel.send(BlockingChannel.scala:103) at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProdu c er.scala:72) - locked <8489cd8> (a java.lang.Object) at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply $ mcV$sp(SyncProducer.scala:103) at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply ( SyncProducer.scala:103) at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply ( SyncProducer.scala:103) at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) at kafka.producer.SyncProducer$$anonfun$send$1.apply$mcV$s
[jira] [Comment Edited] (KAFKA-914) Deadlock between initial rebalance and watcher-triggered rebalances
[ https://issues.apache.org/jira/browse/KAFKA-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145519#comment-15145519 ] Rekha Joshi edited comment on KAFKA-914 at 2/12/16 11:16 PM: - Hi, We have been seeing consistent issue mirroring between our DataCenters., and same issue seems to resurface.Below are the details.Is this concern really resolved? Thanks Rekha {code} Source: AWS (13 Brokers) Destination: OTHER-DC (20 Brokers) Topic: mirrortest (source: 260 partitions, destination: 200 partitions) Connectivity: AWS Direct Connect (max 6Gbps) Data details: Source is receiving 40,000 msg/sec, each message is around 5KB Mirroring Node: is at our OTHER-DC (24 Cores, 48 GB Memory) JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 Launch script: kafka.tools.MirrorMaker --consumer.config consumer.properties --producer.config producer.properties --num.producers 1 --whitelist mirrortest --num.streams 1 --queue.size 10 consumer.properties --- zookeeper.connect= group.id=KafkaMirror auto.offset.reset=smallest fetch.message.max.bytes=900 zookeeper.connection.timeout.ms=6 rebalance.max.retries=4 rebalance.backoff.ms=5000 producer.properties -- metadata.broker.list= partitioner.class= producer.type=async When we start the mirroring job everything works fine as expected, Eventually we hit an issue where the job stops consuming no more. At this stage: 1. No Error seen in the mirrormaker logs 2. consumer threads are not fetching any messages and we see thread dumps as follows: "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread t@73 java.lang.Thread.State: WAITING at sun.misc.Unsafe.park(Native Method) - parking to wait for <79b6d3ce> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa i t(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 ) at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) at kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher T hread.scala:49) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu n $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu n $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m c V$sp(AbstractFetcherThread.scala:109) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A b stractFetcherThread.scala:109) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A b stractFetcherThread.scala:109) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr e ad.scala:108) at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) Locked ownable synchronizers: - locked <199dc92d> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) 3. Producer stops producing, in trace mode we notice it's handling 0 events and Thread dump as follows: "ProducerSendThread--0" - Thread t@53 java.lang.Thread.State: RUNNABLE at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.write(IOUtil.java:148) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) - locked <5ae2fc40> (a java.lang.Object) at java.nio.channels.SocketChannel.write(SocketChannel.java:502) at kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:5 6 ) at kafka.network.Send$class.writeCompletely(Transmission.scala:75) at kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend . scala:26) at kafka.network.BlockingChannel.send(BlockingChannel.scala:103) at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProdu c er.scala:72) - locked <8489cd8> (a java.lang.Object) at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply $ mcV$sp(SyncProducer.scala:103) at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply ( SyncProducer.scala:103) at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply ( SyncProducer.scala:103) at kafka.metrics.KafkaTimer.time(KafkaTimer.
[jira] [Created] (KAFKA-3238) Deadlock Mirrormaker consumer not fetching any messages
Rekha Joshi created KAFKA-3238: -- Summary: Deadlock Mirrormaker consumer not fetching any messages Key: KAFKA-3238 URL: https://issues.apache.org/jira/browse/KAFKA-3238 Project: Kafka Issue Type: Bug Affects Versions: 0.8.0 Reporter: Rekha Joshi Hi, We have been seeing consistent issue mirroring between our DataCenters happening randomly.Below are the details. Thanks Rekha {code} Source: AWS (13 Brokers) Destination: OTHER-DC (20 Brokers) Topic: mirrortest (source: 260 partitions, destination: 200 partitions) Connectivity: AWS Direct Connect (max 6Gbps) Data details: Source is receiving 40,000 msg/sec, each message is around 5KB Mirroring Node: is at our OTHER-DC (24 Cores, 48 GB Memory) JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 Launch script: kafka.tools.MirrorMaker --consumer.config consumer.properties --producer.config producer.properties --num.producers 1 --whitelist mirrortest --num.streams 1 --queue.size 10 consumer.properties --- zookeeper.connect= group.id=KafkaMirror auto.offset.reset=smallest fetch.message.max.bytes=900 zookeeper.connection.timeout.ms=6 rebalance.max.retries=4 rebalance.backoff.ms=5000 producer.properties -- metadata.broker.list= partitioner.class= producer.type=async When we start the mirroring job everything works fine as expected, Eventually we hit an issue where the job stops consuming no more. At this stage: 1. No Error seen in the mirrormaker logs 2. consumer threads are not fetching any messages and we see thread dumps as follows: "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread t@73 java.lang.Thread.State: WAITING at sun.misc.Unsafe.park(Native Method) - parking to wait for <79b6d3ce> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa i t(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 ) at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) at kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher T hread.scala:49) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu n $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu n $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m c V$sp(AbstractFetcherThread.scala:109) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A b stractFetcherThread.scala:109) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A b stractFetcherThread.scala:109) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr e ad.scala:108) at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) Locked ownable synchronizers: - locked <199dc92d> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) 3. Producer stops producing, in trace mode we notice it's handling 0 events and Thread dump as follows: "ProducerSendThread--0" - Thread t@53 java.lang.Thread.State: RUNNABLE at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.write(IOUtil.java:148) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) - locked <5ae2fc40> (a java.lang.Object) at java.nio.channels.SocketChannel.write(SocketChannel.java:502) at kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:5 6 ) at kafka.network.Send$class.writeCompletely(Transmission.scala:75) at kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend . scala:26) at kafka.network.BlockingChannel.send(BlockingChannel.scala:103) at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProdu c er.scala:72) - locked <8489cd8> (a java.lang.Object) at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply $ mcV$sp(SyncProducer.scala:103) at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply ( SyncProducer.scala:103) at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply ( SyncProducer.scala:103) at kafka.metrics.KafkaTim
[jira] [Comment Edited] (KAFKA-914) Deadlock between initial rebalance and watcher-triggered rebalances
[ https://issues.apache.org/jira/browse/KAFKA-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145519#comment-15145519 ] Rekha Joshi edited comment on KAFKA-914 at 2/13/16 1:20 AM: Hi, Facing similar issue; raised in https://issues.apache.org/jira/browse/KAFKA-3238 Thanks Rekha was (Author: rekhajoshm): Hi, We have been seeing consistent issue mirroring between our DataCenters., and same issue seems to resurface.Below are the details.Is this concern really resolved? Thanks Rekha {code} Source: AWS (13 Brokers) Destination: OTHER-DC (20 Brokers) Topic: mirrortest (source: 260 partitions, destination: 200 partitions) Connectivity: AWS Direct Connect (max 6Gbps) Data details: Source is receiving 40,000 msg/sec, each message is around 5KB Mirroring Node: is at our OTHER-DC (24 Cores, 48 GB Memory) JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 Launch script: kafka.tools.MirrorMaker --consumer.config consumer.properties --producer.config producer.properties --num.producers 1 --whitelist mirrortest --num.streams 1 --queue.size 10 consumer.properties --- zookeeper.connect= group.id=KafkaMirror auto.offset.reset=smallest fetch.message.max.bytes=900 zookeeper.connection.timeout.ms=6 rebalance.max.retries=4 rebalance.backoff.ms=5000 producer.properties -- metadata.broker.list= partitioner.class= producer.type=async When we start the mirroring job everything works fine as expected, Eventually we hit an issue where the job stops consuming no more. At this stage: 1. No Error seen in the mirrormaker logs 2. consumer threads are not fetching any messages and we see thread dumps as follows: "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread t@73 java.lang.Thread.State: WAITING at sun.misc.Unsafe.park(Native Method) - parking to wait for <79b6d3ce> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa i t(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 ) at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) at kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher T hread.scala:49) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu n $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu n $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m c V$sp(AbstractFetcherThread.scala:109) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A b stractFetcherThread.scala:109) at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A b stractFetcherThread.scala:109) at kafka.utils.Utils$.inLock(Utils.scala:535) at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr e ad.scala:108) at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) Locked ownable synchronizers: - locked <199dc92d> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) 3. Producer stops producing, in trace mode we notice it's handling 0 events and Thread dump as follows: "ProducerSendThread--0" - Thread t@53 java.lang.Thread.State: RUNNABLE at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.write(IOUtil.java:148) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) - locked <5ae2fc40> (a java.lang.Object) at java.nio.channels.SocketChannel.write(SocketChannel.java:502) at kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:5 6 ) at kafka.network.Send$class.writeCompletely(Transmission.scala:75) at kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend . scala:26) at kafka.network.BlockingChannel.send(BlockingChannel.scala:103) at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProdu c er.scala:72) - locked <8489cd8> (a java.lang.Object) at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply $ mcV$sp(SyncProducer.scala:103) at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply ( SyncProducer.scala:103) at kafka.producer.Sync
[jira] [Updated] (KAFKA-3238) Deadlock Mirrormaker consumer not fetching any messages
[ https://issues.apache.org/jira/browse/KAFKA-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-3238: --- Priority: Critical (was: Major) > Deadlock Mirrormaker consumer not fetching any messages > --- > > Key: KAFKA-3238 > URL: https://issues.apache.org/jira/browse/KAFKA-3238 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.0 >Reporter: Rekha Joshi >Priority: Critical > > Hi, > We have been seeing consistent issue mirroring between our DataCenters > happening randomly.Below are the details. > Thanks > Rekha > {code} > Source: AWS (13 Brokers) > Destination: OTHER-DC (20 Brokers) > Topic: mirrortest (source: 260 partitions, destination: 200 partitions) > Connectivity: AWS Direct Connect (max 6Gbps) > Data details: Source is receiving 40,000 msg/sec, each message is around > 5KB > Mirroring > > Node: is at our OTHER-DC (24 Cores, 48 GB Memory) > JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m > -XX:+UseG1GC -XX:MaxGCPauseMillis=20 > -XX:InitiatingHeapOccupancyPercent=35 > Launch script: kafka.tools.MirrorMaker --consumer.config > consumer.properties --producer.config producer.properties --num.producers > 1 --whitelist mirrortest --num.streams 1 --queue.size 10 > consumer.properties > --- > zookeeper.connect= > group.id=KafkaMirror > auto.offset.reset=smallest > fetch.message.max.bytes=900 > zookeeper.connection.timeout.ms=6 > rebalance.max.retries=4 > rebalance.backoff.ms=5000 > producer.properties > -- > metadata.broker.list= > partitioner.class= > producer.type=async > When we start the mirroring job everything works fine as expected, > Eventually we hit an issue where the job stops consuming no more. > At this stage: > 1. No Error seen in the mirrormaker logs > 2. consumer threads are not fetching any messages and we see thread dumps > as follows: > "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread > t@73 > java.lang.Thread.State: WAITING > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <79b6d3ce> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa > i > t(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 > ) > at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher > T > hread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) > at > scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m > c > V$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:535) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr > e > ad.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > Locked ownable synchronizers: > - locked <199dc92d> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > 3. Producer stops producing, in trace mode we notice it's handling 0 > events and Thread dump as follows: > "ProducerSendThread--0" - Thread t@53 > java.lang.Thread.State: RUNNABLE > at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) > at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) > at sun.nio.ch.IOUtil.write(IOUtil.java:148) > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) > - locked <5ae2fc40> (a java.lang.Object) > at java.nio.channels.SocketChannel.write(SocketChannel.java:502) > at > kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:5 > 6 > ) > at kafka.network.Send$class.writeCompletely(Transmission.scala:75) > at > kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend > . > scala:26) > at kafka.network.BlockingChannel.send(BlockingChannel.scala:103) > at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) > at > kafka.pro
[jira] [Commented] (KAFKA-3238) Deadlock Mirrormaker consumer not fetching any messages
[ https://issues.apache.org/jira/browse/KAFKA-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145786#comment-15145786 ] Rekha Joshi commented on KAFKA-3238: Hi, [~nehanarkhede] [~junrao] [~jkreps] : I have seen few deadlock related issues on kafka - KAFKA-914 , KAFKA-702 and was wondering if it relates to datastructures used/locking pattern/JDK related. For eg: http://bugs.java.com/view_bug.do?bug_id=7011862 Please let me know your analysis/thoughts on the mirrormaker issue here? thanks! Thanks Rekha > Deadlock Mirrormaker consumer not fetching any messages > --- > > Key: KAFKA-3238 > URL: https://issues.apache.org/jira/browse/KAFKA-3238 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.0 >Reporter: Rekha Joshi >Priority: Critical > > Hi, > We have been seeing consistent issue mirroring between our DataCenters > happening randomly.Below are the details. > Thanks > Rekha > {code} > Source: AWS (13 Brokers) > Destination: OTHER-DC (20 Brokers) > Topic: mirrortest (source: 260 partitions, destination: 200 partitions) > Connectivity: AWS Direct Connect (max 6Gbps) > Data details: Source is receiving 40,000 msg/sec, each message is around > 5KB > Mirroring > > Node: is at our OTHER-DC (24 Cores, 48 GB Memory) > JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m > -XX:+UseG1GC -XX:MaxGCPauseMillis=20 > -XX:InitiatingHeapOccupancyPercent=35 > Launch script: kafka.tools.MirrorMaker --consumer.config > consumer.properties --producer.config producer.properties --num.producers > 1 --whitelist mirrortest --num.streams 1 --queue.size 10 > consumer.properties > --- > zookeeper.connect= > group.id=KafkaMirror > auto.offset.reset=smallest > fetch.message.max.bytes=900 > zookeeper.connection.timeout.ms=6 > rebalance.max.retries=4 > rebalance.backoff.ms=5000 > producer.properties > -- > metadata.broker.list= > partitioner.class= > producer.type=async > When we start the mirroring job everything works fine as expected, > Eventually we hit an issue where the job stops consuming no more. > At this stage: > 1. No Error seen in the mirrormaker logs > 2. consumer threads are not fetching any messages and we see thread dumps > as follows: > "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread > t@73 > java.lang.Thread.State: WAITING > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <79b6d3ce> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa > i > t(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 > ) > at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher > T > hread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) > at > scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m > c > V$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:535) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr > e > ad.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > Locked ownable synchronizers: > - locked <199dc92d> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > 3. Producer stops producing, in trace mode we notice it's handling 0 > events and Thread dump as follows: > "ProducerSendThread--0" - Thread t@53 > java.lang.Thread.State: RUNNABLE > at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) > at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) > at sun.nio.ch.IOUtil.write(IOUtil.java:148) > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) > - locked <5ae2fc40> (a java.lang.Object) > at java.nio.channels.SocketChannel.write(SocketChannel.java:502) > at > kafka.network.BoundedByteBu
[jira] [Comment Edited] (KAFKA-3238) Deadlock Mirrormaker consumer not fetching any messages
[ https://issues.apache.org/jira/browse/KAFKA-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145786#comment-15145786 ] Rekha Joshi edited comment on KAFKA-3238 at 2/13/16 4:24 AM: - Hi, [~nehanarkhede] [~junrao] [~jkreps] : I have seen few similar deadlock related issues on kafka - KAFKA-914 , KAFKA-702 and was wondering if it relates to datastructures used/locking pattern/JDK related. For eg: http://bugs.java.com/view_bug.do?bug_id=7011862 Please let me know your analysis/thoughts on the mirrormaker issue here? thanks! Thanks Rekha was (Author: rekhajoshm): Hi, [~nehanarkhede] [~junrao] [~jkreps] : I have seen few deadlock related issues on kafka - KAFKA-914 , KAFKA-702 and was wondering if it relates to datastructures used/locking pattern/JDK related. For eg: http://bugs.java.com/view_bug.do?bug_id=7011862 Please let me know your analysis/thoughts on the mirrormaker issue here? thanks! Thanks Rekha > Deadlock Mirrormaker consumer not fetching any messages > --- > > Key: KAFKA-3238 > URL: https://issues.apache.org/jira/browse/KAFKA-3238 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.0 >Reporter: Rekha Joshi >Priority: Critical > > Hi, > We have been seeing consistent issue mirroring between our DataCenters > happening randomly.Below are the details. > Thanks > Rekha > {code} > Source: AWS (13 Brokers) > Destination: OTHER-DC (20 Brokers) > Topic: mirrortest (source: 260 partitions, destination: 200 partitions) > Connectivity: AWS Direct Connect (max 6Gbps) > Data details: Source is receiving 40,000 msg/sec, each message is around > 5KB > Mirroring > > Node: is at our OTHER-DC (24 Cores, 48 GB Memory) > JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m > -XX:+UseG1GC -XX:MaxGCPauseMillis=20 > -XX:InitiatingHeapOccupancyPercent=35 > Launch script: kafka.tools.MirrorMaker --consumer.config > consumer.properties --producer.config producer.properties --num.producers > 1 --whitelist mirrortest --num.streams 1 --queue.size 10 > consumer.properties > --- > zookeeper.connect= > group.id=KafkaMirror > auto.offset.reset=smallest > fetch.message.max.bytes=900 > zookeeper.connection.timeout.ms=6 > rebalance.max.retries=4 > rebalance.backoff.ms=5000 > producer.properties > -- > metadata.broker.list= > partitioner.class= > producer.type=async > When we start the mirroring job everything works fine as expected, > Eventually we hit an issue where the job stops consuming no more. > At this stage: > 1. No Error seen in the mirrormaker logs > 2. consumer threads are not fetching any messages and we see thread dumps > as follows: > "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread > t@73 > java.lang.Thread.State: WAITING > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <79b6d3ce> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa > i > t(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 > ) > at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher > T > hread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) > at > scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m > c > V$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:535) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr > e > ad.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > Locked ownable synchronizers: > - locked <199dc92d> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > 3. Producer stops producing, in trace mode we notice it's handling 0 > events and Thread dump as follows: > "ProducerSendThread
[jira] [Updated] (KAFKA-3238) Deadlock Mirrormaker consumer not fetching any messages
[ https://issues.apache.org/jira/browse/KAFKA-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-3238: --- Priority: Blocker (was: Critical) > Deadlock Mirrormaker consumer not fetching any messages > --- > > Key: KAFKA-3238 > URL: https://issues.apache.org/jira/browse/KAFKA-3238 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.0 >Reporter: Rekha Joshi >Priority: Blocker > > Hi, > We have been seeing consistent issue mirroring between our DataCenters > happening randomly.Below are the details. > Thanks > Rekha > {code} > Source: AWS (13 Brokers) > Destination: OTHER-DC (20 Brokers) > Topic: mirrortest (source: 260 partitions, destination: 200 partitions) > Connectivity: AWS Direct Connect (max 6Gbps) > Data details: Source is receiving 40,000 msg/sec, each message is around > 5KB > Mirroring > > Node: is at our OTHER-DC (24 Cores, 48 GB Memory) > JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m > -XX:+UseG1GC -XX:MaxGCPauseMillis=20 > -XX:InitiatingHeapOccupancyPercent=35 > Launch script: kafka.tools.MirrorMaker --consumer.config > consumer.properties --producer.config producer.properties --num.producers > 1 --whitelist mirrortest --num.streams 1 --queue.size 10 > consumer.properties > --- > zookeeper.connect= > group.id=KafkaMirror > auto.offset.reset=smallest > fetch.message.max.bytes=900 > zookeeper.connection.timeout.ms=6 > rebalance.max.retries=4 > rebalance.backoff.ms=5000 > producer.properties > -- > metadata.broker.list= > partitioner.class= > producer.type=async > When we start the mirroring job everything works fine as expected, > Eventually we hit an issue where the job stops consuming no more. > At this stage: > 1. No Error seen in the mirrormaker logs > 2. consumer threads are not fetching any messages and we see thread dumps > as follows: > "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread > t@73 > java.lang.Thread.State: WAITING > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <79b6d3ce> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa > i > t(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 > ) > at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher > T > hread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) > at > scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m > c > V$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:535) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr > e > ad.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > Locked ownable synchronizers: > - locked <199dc92d> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > 3. Producer stops producing, in trace mode we notice it's handling 0 > events and Thread dump as follows: > "ProducerSendThread--0" - Thread t@53 > java.lang.Thread.State: RUNNABLE > at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) > at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) > at sun.nio.ch.IOUtil.write(IOUtil.java:148) > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) > - locked <5ae2fc40> (a java.lang.Object) > at java.nio.channels.SocketChannel.write(SocketChannel.java:502) > at > kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:5 > 6 > ) > at kafka.network.Send$class.writeCompletely(Transmission.scala:75) > at > kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend > . > scala:26) > at kafka.network.BlockingChannel.send(BlockingChannel.scala:103) > at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) > at > kafka.pr
[jira] [Updated] (KAFKA-3238) Deadlock Mirrormaker consumer not fetching any messages
[ https://issues.apache.org/jira/browse/KAFKA-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-3238: --- Priority: Critical (was: Blocker) > Deadlock Mirrormaker consumer not fetching any messages > --- > > Key: KAFKA-3238 > URL: https://issues.apache.org/jira/browse/KAFKA-3238 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.0 >Reporter: Rekha Joshi >Priority: Critical > > Hi, > We have been seeing consistent issue mirroring between our DataCenters > happening randomly.Below are the details. > Thanks > Rekha > {code} > Source: AWS (13 Brokers) > Destination: OTHER-DC (20 Brokers) > Topic: mirrortest (source: 260 partitions, destination: 200 partitions) > Connectivity: AWS Direct Connect (max 6Gbps) > Data details: Source is receiving 40,000 msg/sec, each message is around > 5KB > Mirroring > > Node: is at our OTHER-DC (24 Cores, 48 GB Memory) > JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m > -XX:+UseG1GC -XX:MaxGCPauseMillis=20 > -XX:InitiatingHeapOccupancyPercent=35 > Launch script: kafka.tools.MirrorMaker --consumer.config > consumer.properties --producer.config producer.properties --num.producers > 1 --whitelist mirrortest --num.streams 1 --queue.size 10 > consumer.properties > --- > zookeeper.connect= > group.id=KafkaMirror > auto.offset.reset=smallest > fetch.message.max.bytes=900 > zookeeper.connection.timeout.ms=6 > rebalance.max.retries=4 > rebalance.backoff.ms=5000 > producer.properties > -- > metadata.broker.list= > partitioner.class= > producer.type=async > When we start the mirroring job everything works fine as expected, > Eventually we hit an issue where the job stops consuming no more. > At this stage: > 1. No Error seen in the mirrormaker logs > 2. consumer threads are not fetching any messages and we see thread dumps > as follows: > "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread > t@73 > java.lang.Thread.State: WAITING > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <79b6d3ce> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa > i > t(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 > ) > at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher > T > hread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) > at > scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m > c > V$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:535) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr > e > ad.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > Locked ownable synchronizers: > - locked <199dc92d> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > 3. Producer stops producing, in trace mode we notice it's handling 0 > events and Thread dump as follows: > "ProducerSendThread--0" - Thread t@53 > java.lang.Thread.State: RUNNABLE > at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) > at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) > at sun.nio.ch.IOUtil.write(IOUtil.java:148) > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) > - locked <5ae2fc40> (a java.lang.Object) > at java.nio.channels.SocketChannel.write(SocketChannel.java:502) > at > kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:5 > 6 > ) > at kafka.network.Send$class.writeCompletely(Transmission.scala:75) > at > kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend > . > scala:26) > at kafka.network.BlockingChannel.send(BlockingChannel.scala:103) > at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73) > at > kafka.p
[jira] [Comment Edited] (KAFKA-3238) Deadlock Mirrormaker consumer not fetching any messages
[ https://issues.apache.org/jira/browse/KAFKA-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145786#comment-15145786 ] Rekha Joshi edited comment on KAFKA-3238 at 2/13/16 7:29 AM: - Hi, [~nehanarkhede] [~junrao] [~jkreps] : I have seen few similar deadlock related issues on kafka - KAFKA-914 , KAFKA-702 and few open issues in similar vein.Looking forward to hearing your analysis/thoughts on this mirrormaker issue soon. thanks! Thanks Rekha was (Author: rekhajoshm): Hi, [~nehanarkhede] [~junrao] [~jkreps] : I have seen few similar deadlock related issues on kafka - KAFKA-914 , KAFKA-702 and was wondering if it relates to datastructures used/locking pattern/JDK related. For eg: http://bugs.java.com/view_bug.do?bug_id=7011862 Please let me know your analysis/thoughts on the mirrormaker issue here? thanks! Thanks Rekha > Deadlock Mirrormaker consumer not fetching any messages > --- > > Key: KAFKA-3238 > URL: https://issues.apache.org/jira/browse/KAFKA-3238 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.0 >Reporter: Rekha Joshi >Priority: Critical > > Hi, > We have been seeing consistent issue mirroring between our DataCenters > happening randomly.Below are the details. > Thanks > Rekha > {code} > Source: AWS (13 Brokers) > Destination: OTHER-DC (20 Brokers) > Topic: mirrortest (source: 260 partitions, destination: 200 partitions) > Connectivity: AWS Direct Connect (max 6Gbps) > Data details: Source is receiving 40,000 msg/sec, each message is around > 5KB > Mirroring > > Node: is at our OTHER-DC (24 Cores, 48 GB Memory) > JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m > -XX:+UseG1GC -XX:MaxGCPauseMillis=20 > -XX:InitiatingHeapOccupancyPercent=35 > Launch script: kafka.tools.MirrorMaker --consumer.config > consumer.properties --producer.config producer.properties --num.producers > 1 --whitelist mirrortest --num.streams 1 --queue.size 10 > consumer.properties > --- > zookeeper.connect= > group.id=KafkaMirror > auto.offset.reset=smallest > fetch.message.max.bytes=900 > zookeeper.connection.timeout.ms=6 > rebalance.max.retries=4 > rebalance.backoff.ms=5000 > producer.properties > -- > metadata.broker.list= > partitioner.class= > producer.type=async > When we start the mirroring job everything works fine as expected, > Eventually we hit an issue where the job stops consuming no more. > At this stage: > 1. No Error seen in the mirrormaker logs > 2. consumer threads are not fetching any messages and we see thread dumps > as follows: > "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread > t@73 > java.lang.Thread.State: WAITING > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <79b6d3ce> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa > i > t(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 > ) > at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher > T > hread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) > at > scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m > c > V$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:535) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr > e > ad.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > Locked ownable synchronizers: > - locked <199dc92d> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > 3. Producer stops producing, in trace mode we notice it's handling 0 > events and Thread dump as follows: > "ProducerSendThread--0" - Thread t@53 > java.lang.Thread.State: RUNNABLE > at sun.nio.ch.FileDispatcherIm
[jira] [Commented] (KAFKA-3238) Deadlock Mirrormaker consumer not fetching any messages
[ https://issues.apache.org/jira/browse/KAFKA-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149068#comment-15149068 ] Rekha Joshi commented on KAFKA-3238: anyone home? :) would be great to have your inputs! thanks [~nehanarkhede] [~junrao] [~jkreps] > Deadlock Mirrormaker consumer not fetching any messages > --- > > Key: KAFKA-3238 > URL: https://issues.apache.org/jira/browse/KAFKA-3238 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.0 >Reporter: Rekha Joshi >Priority: Critical > > Hi, > We have been seeing consistent issue mirroring between our DataCenters > happening randomly.Below are the details. > Thanks > Rekha > {code} > Source: AWS (13 Brokers) > Destination: OTHER-DC (20 Brokers) > Topic: mirrortest (source: 260 partitions, destination: 200 partitions) > Connectivity: AWS Direct Connect (max 6Gbps) > Data details: Source is receiving 40,000 msg/sec, each message is around > 5KB > Mirroring > > Node: is at our OTHER-DC (24 Cores, 48 GB Memory) > JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m > -XX:+UseG1GC -XX:MaxGCPauseMillis=20 > -XX:InitiatingHeapOccupancyPercent=35 > Launch script: kafka.tools.MirrorMaker --consumer.config > consumer.properties --producer.config producer.properties --num.producers > 1 --whitelist mirrortest --num.streams 1 --queue.size 10 > consumer.properties > --- > zookeeper.connect= > group.id=KafkaMirror > auto.offset.reset=smallest > fetch.message.max.bytes=900 > zookeeper.connection.timeout.ms=6 > rebalance.max.retries=4 > rebalance.backoff.ms=5000 > producer.properties > -- > metadata.broker.list= > partitioner.class= > producer.type=async > When we start the mirroring job everything works fine as expected, > Eventually we hit an issue where the job stops consuming no more. > At this stage: > 1. No Error seen in the mirrormaker logs > 2. consumer threads are not fetching any messages and we see thread dumps > as follows: > "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread > t@73 > java.lang.Thread.State: WAITING > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <79b6d3ce> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa > i > t(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 > ) > at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher > T > hread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) > at > scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m > c > V$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:535) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr > e > ad.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > Locked ownable synchronizers: > - locked <199dc92d> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > 3. Producer stops producing, in trace mode we notice it's handling 0 > events and Thread dump as follows: > "ProducerSendThread--0" - Thread t@53 > java.lang.Thread.State: RUNNABLE > at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) > at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) > at sun.nio.ch.IOUtil.write(IOUtil.java:148) > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) > - locked <5ae2fc40> (a java.lang.Object) > at java.nio.channels.SocketChannel.write(SocketChannel.java:502) > at > kafka.network.BoundedByteBufferSend.writeTo(BoundedByteBufferSend.scala:5 > 6 > ) > at kafka.network.Send$class.writeCompletely(Transmission.scala:75) > at > kafka.network.BoundedByteBufferSend.writeCompletely(BoundedByteBufferSend > . > scala:26) > at kafka.network.BlockingChannel.s
[jira] [Comment Edited] (KAFKA-3238) Deadlock Mirrormaker consumer not fetching any messages
[ https://issues.apache.org/jira/browse/KAFKA-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149068#comment-15149068 ] Rekha Joshi edited comment on KAFKA-3238 at 2/16/16 9:29 PM: - anyone home? :) This has similarity to KAFKA-2048, but as per my investigation did not see the IllegalMonitorStateException in logs.But the underlying fetcher thread to be completely stopped.The trace substantiate that. Its not clearly mentioned but was KAFKA-2048 is fixed in 0.9? would be great to have your inputs! thanks [~nehanarkhede] [~junrao] [~jkreps] was (Author: rekhajoshm): anyone home? :) would be great to have your inputs! thanks [~nehanarkhede] [~junrao] [~jkreps] > Deadlock Mirrormaker consumer not fetching any messages > --- > > Key: KAFKA-3238 > URL: https://issues.apache.org/jira/browse/KAFKA-3238 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.0 >Reporter: Rekha Joshi >Priority: Critical > > Hi, > We have been seeing consistent issue mirroring between our DataCenters > happening randomly.Below are the details. > Thanks > Rekha > {code} > Source: AWS (13 Brokers) > Destination: OTHER-DC (20 Brokers) > Topic: mirrortest (source: 260 partitions, destination: 200 partitions) > Connectivity: AWS Direct Connect (max 6Gbps) > Data details: Source is receiving 40,000 msg/sec, each message is around > 5KB > Mirroring > > Node: is at our OTHER-DC (24 Cores, 48 GB Memory) > JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m > -XX:+UseG1GC -XX:MaxGCPauseMillis=20 > -XX:InitiatingHeapOccupancyPercent=35 > Launch script: kafka.tools.MirrorMaker --consumer.config > consumer.properties --producer.config producer.properties --num.producers > 1 --whitelist mirrortest --num.streams 1 --queue.size 10 > consumer.properties > --- > zookeeper.connect= > group.id=KafkaMirror > auto.offset.reset=smallest > fetch.message.max.bytes=900 > zookeeper.connection.timeout.ms=6 > rebalance.max.retries=4 > rebalance.backoff.ms=5000 > producer.properties > -- > metadata.broker.list= > partitioner.class= > producer.type=async > When we start the mirroring job everything works fine as expected, > Eventually we hit an issue where the job stops consuming no more. > At this stage: > 1. No Error seen in the mirrormaker logs > 2. consumer threads are not fetching any messages and we see thread dumps > as follows: > "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread > t@73 > java.lang.Thread.State: WAITING > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <79b6d3ce> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa > i > t(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 > ) > at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher > T > hread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) > at > scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m > c > V$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:535) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr > e > ad.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > Locked ownable synchronizers: > - locked <199dc92d> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > 3. Producer stops producing, in trace mode we notice it's handling 0 > events and Thread dump as follows: > "ProducerSendThread--0" - Thread t@53 > java.lang.Thread.State: RUNNABLE > at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) > at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) > at sun.nio.ch.IOUtil.write(IOUtil.java:148) > at sun.nio.ch.SocketCha
[jira] [Comment Edited] (KAFKA-3238) Deadlock Mirrormaker consumer not fetching any messages
[ https://issues.apache.org/jira/browse/KAFKA-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149068#comment-15149068 ] Rekha Joshi edited comment on KAFKA-3238 at 2/17/16 3:46 AM: - anyone home? :) This has similarity to KAFKA-2048, but as per my investigation did not see the IllegalMonitorStateException in logs.The trace here suggest the underlying fetcher thread is stopped.Thankfully for having logs issue corrected, KAFKA-1891 is fixed in Kafka 0.9. Maybe with mirrormaker refactored in 0.9 and few other fixes, Kafka 0.9 is one option? Anyhow it requires uphaul of a production system at my end, so wondering if there are any other suggestions? Would be great to have your inputs! thanks [~nehanarkhede] [~junrao] [~jkreps] was (Author: rekhajoshm): anyone home? :) This has similarity to KAFKA-2048, but as per my investigation did not see the IllegalMonitorStateException in logs.But the underlying fetcher thread to be completely stopped.The trace substantiate that. Its not clearly mentioned but was KAFKA-2048 is fixed in 0.9? would be great to have your inputs! thanks [~nehanarkhede] [~junrao] [~jkreps] > Deadlock Mirrormaker consumer not fetching any messages > --- > > Key: KAFKA-3238 > URL: https://issues.apache.org/jira/browse/KAFKA-3238 > Project: Kafka > Issue Type: Bug >Affects Versions: 0.8.0 >Reporter: Rekha Joshi >Priority: Critical > > Hi, > We have been seeing consistent issue mirroring between our DataCenters > happening randomly.Below are the details. > Thanks > Rekha > {code} > Source: AWS (13 Brokers) > Destination: OTHER-DC (20 Brokers) > Topic: mirrortest (source: 260 partitions, destination: 200 partitions) > Connectivity: AWS Direct Connect (max 6Gbps) > Data details: Source is receiving 40,000 msg/sec, each message is around > 5KB > Mirroring > > Node: is at our OTHER-DC (24 Cores, 48 GB Memory) > JVM Details: 1.8.0_51, -Xmx8G -Xms8G -XX:PermSize=96m -XX:MaxPermSize=96m > -XX:+UseG1GC -XX:MaxGCPauseMillis=20 > -XX:InitiatingHeapOccupancyPercent=35 > Launch script: kafka.tools.MirrorMaker --consumer.config > consumer.properties --producer.config producer.properties --num.producers > 1 --whitelist mirrortest --num.streams 1 --queue.size 10 > consumer.properties > --- > zookeeper.connect= > group.id=KafkaMirror > auto.offset.reset=smallest > fetch.message.max.bytes=900 > zookeeper.connection.timeout.ms=6 > rebalance.max.retries=4 > rebalance.backoff.ms=5000 > producer.properties > -- > metadata.broker.list= > partitioner.class= > producer.type=async > When we start the mirroring job everything works fine as expected, > Eventually we hit an issue where the job stops consuming no more. > At this stage: > 1. No Error seen in the mirrormaker logs > 2. consumer threads are not fetching any messages and we see thread dumps > as follows: > "ConsumerFetcherThread-KafkaMirror_1454375262613-7b760d87-0-26" - Thread > t@73 > java.lang.Thread.State: WAITING > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <79b6d3ce> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awa > i > t(AbstractQueuedSynchronizer.java:2039) > at > java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:350 > ) > at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60) > at > kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcher > T > hread.scala:49) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfu > n > $apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109) > at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) > at > scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$m > c > V$sp(AbstractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at > kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(A > b > stractFetcherThread.scala:109) > at kafka.utils.Utils$.inLock(Utils.scala:535) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThr > e > ad.scala:108) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) > Locked ownable synchronizers: >
[jira] [Created] (KAFKA-3756) javadoc has issues with incorrect @param, @throws, @return
Rekha Joshi created KAFKA-3756: -- Summary: javadoc has issues with incorrect @param, @throws, @return Key: KAFKA-3756 URL: https://issues.apache.org/jira/browse/KAFKA-3756 Project: Kafka Issue Type: Improvement Components: packaging Affects Versions: 0.8.2.0 Reporter: Rekha Joshi Assignee: Rekha Joshi Priority: Minor javadoc has issues with incorrect @param, @throws, @return -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-3765) Code style issues in Kafka
Rekha Joshi created KAFKA-3765: -- Summary: Code style issues in Kafka Key: KAFKA-3765 URL: https://issues.apache.org/jira/browse/KAFKA-3765 Project: Kafka Issue Type: Improvement Components: core Affects Versions: 0.10.0.0 Reporter: Rekha Joshi Assignee: Rekha Joshi Priority: Minor Code style issues in Kafka trunk -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-3771) Improving Kafka code
Rekha Joshi created KAFKA-3771: -- Summary: Improving Kafka code Key: KAFKA-3771 URL: https://issues.apache.org/jira/browse/KAFKA-3771 Project: Kafka Issue Type: Improvement Components: core Affects Versions: 0.10.0.0 Reporter: Rekha Joshi Assignee: Rekha Joshi Improve Kafka core code : Remove redundant val modifier for case class constructor Use flatMap instead of map and flatten Use isEmpty, NonEmpty, isDefined as appropriate Use head, keys and keySet where appropriate Use contains, diff and find where appropriate toString has no parameters, no side effect hence without () use consistently Remove unnecessary return and semi colons, parentheses -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-724) Allow automatic socket.send.buffer from operating system
[ https://issues.apache.org/jira/browse/KAFKA-724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi reassigned KAFKA-724: - Assignee: Rekha Joshi > Allow automatic socket.send.buffer from operating system > > > Key: KAFKA-724 > URL: https://issues.apache.org/jira/browse/KAFKA-724 > Project: Kafka > Issue Type: Improvement >Affects Versions: 0.8.2.0 >Reporter: Pablo Barrera >Assignee: Rekha Joshi > Labels: newbie > > To do this, don't call to socket().setXXXBufferSize. This can be > controlled by the configuration parameter: if the value socket.send.buffer or > others are set to -1, don't call to socket().setXXXBufferSize -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-3796) SslTransportLayerTest.testInvalidEndpointIdentification fails on trunk
Rekha Joshi created KAFKA-3796: -- Summary: SslTransportLayerTest.testInvalidEndpointIdentification fails on trunk Key: KAFKA-3796 URL: https://issues.apache.org/jira/browse/KAFKA-3796 Project: Kafka Issue Type: Bug Components: clients, security Affects Versions: 0.10.0.0 Reporter: Rekha Joshi Assignee: Rekha Joshi org.apache.kafka.common.network.SslTransportLayerTest > testEndpointIdentificationDisabled FAILED java.net.BindException: Can't assign requested address at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:433) at sun.nio.ch.Net.bind(Net.java:425) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67) at org.apache.kafka.common.network.NioEchoServer.(NioEchoServer.java:48) at org.apache.kafka.common.network.SslTransportLayerTest.testEndpointIdentificationDisabled(SslTransportLayerTest.java:120) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-3061) Get rid of Guava dependency
[ https://issues.apache.org/jira/browse/KAFKA-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi reassigned KAFKA-3061: -- Assignee: Rekha Joshi > Get rid of Guava dependency > --- > > Key: KAFKA-3061 > URL: https://issues.apache.org/jira/browse/KAFKA-3061 > Project: Kafka > Issue Type: Bug >Reporter: Gwen Shapira >Assignee: Rekha Joshi > > KAFKA-2422 adds Reflections library to KafkaConnect, which depends on Guava. > Since lots of people want to use Guavas, having it in the framework will lead > to conflicts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3796) SslTransportLayerTest.testInvalidEndpointIdentification fails on trunk
[ https://issues.apache.org/jira/browse/KAFKA-3796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15319236#comment-15319236 ] Rekha Joshi commented on KAFKA-3796: Actually strangely I was not able to replicate this next day, with same build and with latest trunk build.So ofcourse inconsistent, but maybe something to do with my computer at the time.Anyhow, I will keep it open for couple of days, if I never notice this again, will close it then. HTW. Thanks [~ijuma] > SslTransportLayerTest.testInvalidEndpointIdentification fails on trunk > -- > > Key: KAFKA-3796 > URL: https://issues.apache.org/jira/browse/KAFKA-3796 > Project: Kafka > Issue Type: Bug > Components: clients, security >Affects Versions: 0.10.0.0 >Reporter: Rekha Joshi >Assignee: Rekha Joshi > > org.apache.kafka.common.network.SslTransportLayerTest > > testEndpointIdentificationDisabled FAILED > java.net.BindException: Can't assign requested address > at sun.nio.ch.Net.bind0(Native Method) > at sun.nio.ch.Net.bind(Net.java:433) > at sun.nio.ch.Net.bind(Net.java:425) > at > sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67) > at > org.apache.kafka.common.network.NioEchoServer.(NioEchoServer.java:48) > at > org.apache.kafka.common.network.SslTransportLayerTest.testEndpointIdentificationDisabled(SslTransportLayerTest.java:120) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-3844) Sort configuration items in log
[ https://issues.apache.org/jira/browse/KAFKA-3844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi reassigned KAFKA-3844: -- Assignee: Rekha Joshi > Sort configuration items in log > --- > > Key: KAFKA-3844 > URL: https://issues.apache.org/jira/browse/KAFKA-3844 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.10.0.0 >Reporter: Xing Huang >Assignee: Rekha Joshi >Priority: Trivial > > Currently, the output of > org.apache.kafka.common.config.AbstractConfig#logAll() is unsorted, so it's > not convenient to check related configurations. The configuration items in > log could be sorted, so that related items be adjacent. For example, all > "log.*" configuration items would be adjacent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-3905) remove null from subscribed topics in KafkaConsumer#subscribe
[ https://issues.apache.org/jira/browse/KAFKA-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi reassigned KAFKA-3905: -- Assignee: Rekha Joshi > remove null from subscribed topics in KafkaConsumer#subscribe > -- > > Key: KAFKA-3905 > URL: https://issues.apache.org/jira/browse/KAFKA-3905 > Project: Kafka > Issue Type: Wish > Components: clients >Affects Versions: 0.10.0.0 >Reporter: Xing Huang >Assignee: Rekha Joshi >Priority: Minor > > Currently, KafkaConsumer's subscribe methods accept Collection as > topics to be subscribed, but a Collection may have null as its element. For > example > {code} > String topic = null; > Collection topics = Arrays.asList(topic); > consumer.subscribe(topics) > {code} > When this happens, consumer will throw a puzzling NullPointerException: > {code} > at org.apache.kafka.common.utils.Utils.utf8Length(Utils.java:245) > at org.apache.kafka.common.protocol.types.Type$6.sizeOf(Type.java:248) > at > org.apache.kafka.common.protocol.types.ArrayOf.sizeOf(ArrayOf.java:85) > at org.apache.kafka.common.protocol.types.Schema.sizeOf(Schema.java:89) > at org.apache.kafka.common.protocol.types.Struct.sizeOf(Struct.java:244) > at > org.apache.kafka.common.requests.RequestSend.serialize(RequestSend.java:35) > at > org.apache.kafka.common.requests.RequestSend.(RequestSend.java:29) > at > org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.request(NetworkClient.java:616) > at > org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:639) > at > org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate(NetworkClient.java:552) > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:258) > at > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:360) > at > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:224) > at > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:192) > at > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:163) > at > org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:179) > at > org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:970) > at > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:934) > {code} > Maybe it's better to remove null when doing subscription. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-4010) ConfigDef.toRst() should create sections for each group
[ https://issues.apache.org/jira/browse/KAFKA-4010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi reassigned KAFKA-4010: -- Assignee: Rekha Joshi > ConfigDef.toRst() should create sections for each group > --- > > Key: KAFKA-4010 > URL: https://issues.apache.org/jira/browse/KAFKA-4010 > Project: Kafka > Issue Type: Improvement >Reporter: Shikhar Bhushan >Assignee: Rekha Joshi >Priority: Minor > > Currently the ordering seems a bit arbitrary. There is a logical grouping > that connectors are now able to specify with the 'group' field, which we > should use as section headers. Also it would be good to generate {{:ref:}} > for each section. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KAFKA-4014) Use Collections.singletonList instead of Arrays.asList for unmodified single arguments
[ https://issues.apache.org/jira/browse/KAFKA-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi reassigned KAFKA-4014: -- Assignee: Rekha Joshi > Use Collections.singletonList instead of Arrays.asList for unmodified single > arguments > -- > > Key: KAFKA-4014 > URL: https://issues.apache.org/jira/browse/KAFKA-4014 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.9.0.1 >Reporter: Rekha Joshi >Assignee: Rekha Joshi >Priority: Minor > > Usage of Collections.singletonList instead of Arrays.asList for single > arguments better -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-4014) Use Collections.singletonList instead of Arrays.asList for unmodified single arguments
[ https://issues.apache.org/jira/browse/KAFKA-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-4014: --- Issue Type: Improvement (was: Bug) > Use Collections.singletonList instead of Arrays.asList for unmodified single > arguments > -- > > Key: KAFKA-4014 > URL: https://issues.apache.org/jira/browse/KAFKA-4014 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.9.0.1 >Reporter: Rekha Joshi >Priority: Minor > > Usage of Collections.singletonList instead of Arrays.asList for single > arguments better -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-4014) Use Collections.singletonList instead of Arrays.asList for unmodified single arguments
Rekha Joshi created KAFKA-4014: -- Summary: Use Collections.singletonList instead of Arrays.asList for unmodified single arguments Key: KAFKA-4014 URL: https://issues.apache.org/jira/browse/KAFKA-4014 Project: Kafka Issue Type: Bug Components: clients Affects Versions: 0.9.0.1 Reporter: Rekha Joshi Priority: Minor Usage of Collections.singletonList instead of Arrays.asList for single arguments better -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-4014) Use Collections.singletonList instead of Arrays.asList for single arguments
[ https://issues.apache.org/jira/browse/KAFKA-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-4014: --- Description: Usage of Collections.singletonList instead of Arrays.asList for single arguments better for performance (was: Usage of Collections.singletonList instead of Arrays.asList for single arguments better) > Use Collections.singletonList instead of Arrays.asList for single arguments > --- > > Key: KAFKA-4014 > URL: https://issues.apache.org/jira/browse/KAFKA-4014 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.9.0.1 >Reporter: Rekha Joshi >Assignee: Rekha Joshi >Priority: Minor > > Usage of Collections.singletonList instead of Arrays.asList for single > arguments better for performance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-4014) Use Collections.singletonList instead of Arrays.asList for single arguments
[ https://issues.apache.org/jira/browse/KAFKA-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated KAFKA-4014: --- Summary: Use Collections.singletonList instead of Arrays.asList for single arguments (was: Use Collections.singletonList instead of Arrays.asList for unmodified single arguments) > Use Collections.singletonList instead of Arrays.asList for single arguments > --- > > Key: KAFKA-4014 > URL: https://issues.apache.org/jira/browse/KAFKA-4014 > Project: Kafka > Issue Type: Improvement > Components: clients >Affects Versions: 0.9.0.1 >Reporter: Rekha Joshi >Assignee: Rekha Joshi >Priority: Minor > > Usage of Collections.singletonList instead of Arrays.asList for single > arguments better -- This message was sent by Atlassian JIRA (v6.3.4#6332)