[jira] [Resolved] (KAFKA-5571) Possible deadlock during shutdown in setState in kafka streams 10.2
[ https://issues.apache.org/jira/browse/KAFKA-5571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska resolved KAFKA-5571. - Resolution: Fixed > Possible deadlock during shutdown in setState in kafka streams 10.2 > --- > > Key: KAFKA-5571 > URL: https://issues.apache.org/jira/browse/KAFKA-5571 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 0.10.2.1 >Reporter: Greg Fodor >Assignee: Eno Thereska > Attachments: kafka-streams.deadlock.log > > > I'm running a 10.2 job across 5 nodes with 32 stream threads on each node and > find that when gracefully shutdown all of them at once via an ansible > scripts, some of the nodes end up freezing -- at a glance the attached thread > dump implies a deadlock between stream threads trying to update their state > via setState. We haven't had this problem before but it may or may not be > related to changes in 10.2 (we are upgrading from 10.0 to 10.2) > when we gracefully shutdown all nodes simultaneously, what typically happens > is some subset of the nodes end up not shutting down completely but end up > going through a rebalance first. it seems this deadlock requires this > rebalancing to occur simultaneously with the graceful shutdown. if we happen > to shut them down and no rebalance happens, i don't believe this deadlock is > triggered. > the deadlock appears related to the state change handlers being subscribed > across threads and the fact that both StreamThread#setState and > StreamStateListener#onChange are both synchronized methods. > Another thing worth mentioning is that one of the transformers used in the > job has a close() method that can take 10-15 seconds to finish since it needs > to flush some data to a database. Having a long close() method combined with > a rebalance during a shutdown across many threads may be necessary for > reproduction. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-4189) Consumer poll hangs forever if kafka is disabled
[ https://issues.apache.org/jira/browse/KAFKA-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134343#comment-16134343 ] Bhaskar Gollapudi commented on KAFKA-4189: -- Any updates on if wakeup() helped get over the issue ? > Consumer poll hangs forever if kafka is disabled > > > Key: KAFKA-4189 > URL: https://issues.apache.org/jira/browse/KAFKA-4189 > Project: Kafka > Issue Type: Bug > Components: clients, consumer >Affects Versions: 0.9.0.1, 0.10.0.1 >Reporter: Tomas Benc >Priority: Critical > > We develop web application, where client sends REST request and our > application downloads messages from Kafka and sends those messages back to > client. In our web application we use "New Consumer API" (not High Level nor > Simple Consumer API). > Problem occurs in case of disabling Kafka and web application is running on. > Application receives request and tries to poll messages from Kafka. > Processing is on that line blocked until Kafka is enabled. > ConsumerRecords records = consumer.poll(1000); > Timeout parameter of the poll method has no influence in such case. I expect > poll method could throw some Exception describing about connection issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-4189) Consumer poll hangs forever if kafka is disabled
[ https://issues.apache.org/jira/browse/KAFKA-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134497#comment-16134497 ] Tomas Benc commented on KAFKA-4189: --- Yes, wakeup() method helped a lot, it is not issue anymore. > Consumer poll hangs forever if kafka is disabled > > > Key: KAFKA-4189 > URL: https://issues.apache.org/jira/browse/KAFKA-4189 > Project: Kafka > Issue Type: Bug > Components: clients, consumer >Affects Versions: 0.9.0.1, 0.10.0.1 >Reporter: Tomas Benc >Priority: Critical > > We develop web application, where client sends REST request and our > application downloads messages from Kafka and sends those messages back to > client. In our web application we use "New Consumer API" (not High Level nor > Simple Consumer API). > Problem occurs in case of disabling Kafka and web application is running on. > Application receives request and tries to poll messages from Kafka. > Processing is on that line blocked until Kafka is enabled. > ConsumerRecords records = consumer.poll(1000); > Timeout parameter of the poll method has no influence in such case. I expect > poll method could throw some Exception describing about connection issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KAFKA-4189) Consumer poll hangs forever if kafka is disabled
[ https://issues.apache.org/jira/browse/KAFKA-4189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomas Benc resolved KAFKA-4189. --- Resolution: Not A Problem Kafka API contains wakeup() method for such situation. It requires little bit more work - we have to monitor kafka consumer in separate thread, and then call wakeup() from that thread. > Consumer poll hangs forever if kafka is disabled > > > Key: KAFKA-4189 > URL: https://issues.apache.org/jira/browse/KAFKA-4189 > Project: Kafka > Issue Type: Bug > Components: clients, consumer >Affects Versions: 0.9.0.1, 0.10.0.1 >Reporter: Tomas Benc >Priority: Critical > > We develop web application, where client sends REST request and our > application downloads messages from Kafka and sends those messages back to > client. In our web application we use "New Consumer API" (not High Level nor > Simple Consumer API). > Problem occurs in case of disabling Kafka and web application is running on. > Application receives request and tries to poll messages from Kafka. > Processing is on that line blocked until Kafka is enabled. > ConsumerRecords records = consumer.poll(1000); > Timeout parameter of the poll method has no influence in such case. I expect > poll method could throw some Exception describing about connection issues. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KAFKA-5756) Synchronization issue on flush
Oleg Kuznetsov created KAFKA-5756: - Summary: Synchronization issue on flush Key: KAFKA-5756 URL: https://issues.apache.org/jira/browse/KAFKA-5756 Project: Kafka Issue Type: Bug Components: KafkaConnect Affects Versions: 0.10.2.0 Reporter: Oleg Kuznetsov Access to OffsetStorageWriter#toFlush is not synchronized in doFlush() method, whereas this collection can be accessed from 2 different threads: - WorkerSourceTask.execute(), finally block - SourceTaskOffsetCommitter, from periodic flushing task -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KAFKA-5756) Synchronization issue on flush
[ https://issues.apache.org/jira/browse/KAFKA-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kuznetsov updated KAFKA-5756: -- Description: Access to *OffsetStorageWriter#toFlush* is not synchronized in *doFlush()* method, whereas this collection can be accessed from 2 different threads: - *WorkerSourceTask.execute()*, finally block - *SourceTaskOffsetCommitter*, from periodic flushing task was: Access to OffsetStorageWriter#toFlush is not synchronized in doFlush() method, whereas this collection can be accessed from 2 different threads: - WorkerSourceTask.execute(), finally block - SourceTaskOffsetCommitter, from periodic flushing task > Synchronization issue on flush > -- > > Key: KAFKA-5756 > URL: https://issues.apache.org/jira/browse/KAFKA-5756 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.2.0 >Reporter: Oleg Kuznetsov > > Access to *OffsetStorageWriter#toFlush* is not synchronized in *doFlush()* > method, whereas this collection can be accessed from 2 different threads: > - *WorkerSourceTask.execute()*, finally block > - *SourceTaskOffsetCommitter*, from periodic flushing task -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5756) Synchronization issue on flush
[ https://issues.apache.org/jira/browse/KAFKA-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134573#comment-16134573 ] ASF GitHub Bot commented on KAFKA-5756: --- GitHub user oleg-smith opened a pull request: https://github.com/apache/kafka/pull/3702 KAFKA-5756 Synchronization issue on flush You can merge this pull request into a Git repository by running: $ git pull https://github.com/oleg-smith/kafka KAFKA-5756 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3702.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3702 commit 52eb748105fad16e7f810fdedc1a91dcb96b9117 Author: oleg Date: 2017-08-20T21:59:35Z KAFKA-5756 Synchronization issue on flush > Synchronization issue on flush > -- > > Key: KAFKA-5756 > URL: https://issues.apache.org/jira/browse/KAFKA-5756 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.2.0 >Reporter: Oleg Kuznetsov > > Access to *OffsetStorageWriter#toFlush* is not synchronized in *doFlush()* > method, whereas this collection can be accessed from 2 different threads: > - *WorkerSourceTask.execute()*, finally block > - *SourceTaskOffsetCommitter*, from periodic flushing task -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KAFKA-5756) Synchronization issue on flush
[ https://issues.apache.org/jira/browse/KAFKA-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kuznetsov updated KAFKA-5756: -- Description: Access to *OffsetStorageWriter#toFlush* is not synchronized in *doFlush()* method, whereas this collection can be accessed from 2 different threads: - *WorkerSourceTask.execute()*, finally block - *SourceTaskOffsetCommitter*, from periodic flush task was: Access to *OffsetStorageWriter#toFlush* is not synchronized in *doFlush()* method, whereas this collection can be accessed from 2 different threads: - *WorkerSourceTask.execute()*, finally block - *SourceTaskOffsetCommitter*, from periodic flushing task > Synchronization issue on flush > -- > > Key: KAFKA-5756 > URL: https://issues.apache.org/jira/browse/KAFKA-5756 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.2.0 >Reporter: Oleg Kuznetsov > > Access to *OffsetStorageWriter#toFlush* is not synchronized in *doFlush()* > method, whereas this collection can be accessed from 2 different threads: > - *WorkerSourceTask.execute()*, finally block > - *SourceTaskOffsetCommitter*, from periodic flush task -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5756) Synchronization issue on flush
[ https://issues.apache.org/jira/browse/KAFKA-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134575#comment-16134575 ] Oleg Kuznetsov commented on KAFKA-5756: --- [~ewencp] [~rhauch] Could you take a look? > Synchronization issue on flush > -- > > Key: KAFKA-5756 > URL: https://issues.apache.org/jira/browse/KAFKA-5756 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.2.0 >Reporter: Oleg Kuznetsov > > Access to *OffsetStorageWriter#toFlush* is not synchronized in *doFlush()* > method, whereas this collection can be accessed from 2 different threads: > - *WorkerSourceTask.execute()*, finally block > - *SourceTaskOffsetCommitter*, from periodic flush task -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KAFKA-5070) org.apache.kafka.streams.errors.LockException: task [0_18] Failed to lock the state directory: /opt/rocksdb/pulse10/0_18
[ https://issues.apache.org/jira/browse/KAFKA-5070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-5070. Resolution: Duplicate > org.apache.kafka.streams.errors.LockException: task [0_18] Failed to lock the > state directory: /opt/rocksdb/pulse10/0_18 > > > Key: KAFKA-5070 > URL: https://issues.apache.org/jira/browse/KAFKA-5070 > Project: Kafka > Issue Type: Bug > Components: streams >Affects Versions: 0.10.2.0 > Environment: Linux Version >Reporter: Dhana >Assignee: Matthias J. Sax > Attachments: RocksDB_LockStateDirec.7z > > > Notes: we run two instance of consumer in two difference machines/nodes. > we have 400 partitions. 200 stream threads/consumer, with 2 consumer. > We perform HA test(on rebalance - shutdown of one of the consumer/broker), we > see this happening > Error: > 2017-04-05 11:36:09.352 WARN StreamThread:1184 StreamThread-66 - Could not > create task 0_115. Will retry. > org.apache.kafka.streams.errors.LockException: task [0_115] Failed to lock > the state directory: /opt/rocksdb/pulse10/0_115 > at > org.apache.kafka.streams.processor.internals.ProcessorStateManager.(ProcessorStateManager.java:102) > at > org.apache.kafka.streams.processor.internals.AbstractTask.(AbstractTask.java:73) > at > org.apache.kafka.streams.processor.internals.StreamTask.(StreamTask.java:108) > at > org.apache.kafka.streams.processor.internals.StreamThread.createStreamTask(StreamThread.java:834) > at > org.apache.kafka.streams.processor.internals.StreamThread$TaskCreator.createTask(StreamThread.java:1207) > at > org.apache.kafka.streams.processor.internals.StreamThread$AbstractTaskCreator.retryWithBackoff(StreamThread.java:1180) > at > org.apache.kafka.streams.processor.internals.StreamThread.addStreamTasks(StreamThread.java:937) > at > org.apache.kafka.streams.processor.internals.StreamThread.access$500(StreamThread.java:69) > at > org.apache.kafka.streams.processor.internals.StreamThread$1.onPartitionsAssigned(StreamThread.java:236) > at > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:255) > at > org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:339) > at > org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:303) > at > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:286) > at > org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1030) > at > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:995) > at > org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:582) > at > org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:368) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (KAFKA-5744) ShellTest: add tests for attempting to run nonexistent program, error return
[ https://issues.apache.org/jira/browse/KAFKA-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huxihx reopened KAFKA-5744: --- [~cmccabe] On Mac OS, `head -c 0 ` fails with the error "illegal byte count -- 0" instead of "No such file" Could use `head -c 1` > ShellTest: add tests for attempting to run nonexistent program, error return > > > Key: KAFKA-5744 > URL: https://issues.apache.org/jira/browse/KAFKA-5744 > Project: Kafka > Issue Type: Improvement > Components: unit tests >Reporter: Colin P. McCabe >Assignee: Colin P. McCabe > Fix For: 1.0.0 > > > ShellTest should have tests for attempting to run nonexistent program, and > running a program which returns an error. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KAFKA-5744) ShellTest: add tests for attempting to run nonexistent program, error return
[ https://issues.apache.org/jira/browse/KAFKA-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134644#comment-16134644 ] huxihx edited comment on KAFKA-5744 at 8/21/17 2:41 AM: [~cmccabe] [~ijuma] On Mac OS, `head -c 0 ` fails with the error "illegal byte count -- 0" instead of "No such file" Could use `head -c 1` was (Author: huxi_2b): [~cmccabe] On Mac OS, `head -c 0 ` fails with the error "illegal byte count -- 0" instead of "No such file" Could use `head -c 1` > ShellTest: add tests for attempting to run nonexistent program, error return > > > Key: KAFKA-5744 > URL: https://issues.apache.org/jira/browse/KAFKA-5744 > Project: Kafka > Issue Type: Improvement > Components: unit tests >Reporter: Colin P. McCabe >Assignee: Colin P. McCabe > Fix For: 1.0.0 > > > ShellTest should have tests for attempting to run nonexistent program, and > running a program which returns an error. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KAFKA-5755) Refactor Producer to use LogContext
[
https://issues.apache.org/jira/browse/KAFKA-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huxihx reassigned KAFKA-5755:
-
Assignee: huxihx
> Refactor Producer to use LogContext
> ---
>
> Key: KAFKA-5755
> URL: https://issues.apache.org/jira/browse/KAFKA-5755
> Project: Kafka
> Issue Type: Improvement
> Components: producer
>Reporter: Jason Gustafson
>Assignee: huxihx
> Labels: newbie
>
> We added a {{LogContext}} object which automatically adds a log prefix to
> every message written by loggers constructed from it (much like the Logging
> mixin available in the server code). We use this in the consumer to ensure
> that messages always contain the consumer group and client ids, which is very
> helpful when multiple consumers are run on the same instance. We should do
> something similar for the producer. We should always include the client id,
> and if one is provided, the transactional id.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KAFKA-5755) Refactor Producer to use LogContext
[
https://issues.apache.org/jira/browse/KAFKA-5755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134656#comment-16134656
]
ASF GitHub Bot commented on KAFKA-5755:
---
GitHub user huxihx opened a pull request:
https://github.com/apache/kafka/pull/3703
KAFKA-5755: KafkaProducer should be refactored to use LogContext
With LogContext, each producer log item is automatically prefixed with
client id and transactional id.
@hachikuji
Since a new parameter `logContext` is passed into `Sender` constructor
which breaks the max parameter count in checkstyle, I also increased it to 14
now.
And since all the static methods where `log` is used are private in
KafkaProducer, I did not check the nullness for it in those methods.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/huxihx/kafka KAFKA-5755
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/3703.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3703
commit 099c2f9592fc7d137b9f08fac75017e1f00e9323
Author: huxihx
Date: 2017-08-21T02:57:31Z
KAFKA-5755: KafkaProducer should use LogContext instead of the default
LoggerFactory.
With LogContext, each producer log item is automatically prefixed with
client id and transactional id.
> Refactor Producer to use LogContext
> ---
>
> Key: KAFKA-5755
> URL: https://issues.apache.org/jira/browse/KAFKA-5755
> Project: Kafka
> Issue Type: Improvement
> Components: producer
>Reporter: Jason Gustafson
> Labels: newbie
>
> We added a {{LogContext}} object which automatically adds a log prefix to
> every message written by loggers constructed from it (much like the Logging
> mixin available in the server code). We use this in the consumer to ensure
> that messages always contain the consumer group and client ids, which is very
> helpful when multiple consumers are run on the same instance. We should do
> something similar for the producer. We should always include the client id,
> and if one is provided, the transactional id.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KAFKA-5724) AbstractPartitionAssignor does not take into consideration that partition number may start from non-zero
[ https://issues.apache.org/jira/browse/KAFKA-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134658#comment-16134658 ] huxihx commented on KAFKA-5724: --- [~hachikuji] Did you get any time to look into this jira? Two possible ways to solve this problem: 1. Strictly honor the rule that partition must start from zero; 2. Pass more information than partition count when invoking `assign` , in doing so can we correctly do the assignment based on the metadata. What do you think? > AbstractPartitionAssignor does not take into consideration that partition > number may start from non-zero > > > Key: KAFKA-5724 > URL: https://issues.apache.org/jira/browse/KAFKA-5724 > Project: Kafka > Issue Type: Bug > Components: clients, consumer >Affects Versions: 0.11.0.0 >Reporter: Allen Wang >Assignee: huxihx > > In AbstractPartitionAssignor.assign(Cluster metadata, Map Subscription> subscriptions), it invokes assign(partitionsPerTopic, > subscriptions). It assumes that partition number starts from 0, and it > constructs TopicPartition in the range of [0, partitionsPerTopic). > This assumption is not correct. The correct way to handle it is to follow the > same approach in producer's DefaultPartitioner, where it uses [0, > numberOfPartition) as an index to the actual partition. > There are use cases where partition number may not start from zero. It can > happen if users use advanced tooling to manually craft the partition number > when creating topics. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5753) ShellTest.testRunProgramWithErrorReturn fails locally
[
https://issues.apache.org/jira/browse/KAFKA-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134660#comment-16134660
]
huxihx commented on KAFKA-5753:
---
Seems it failed only on Mac OS. Do not use `head -c 0 ...`. Could call `head -c
1 ...` instead. Reopened
[KAFKA-5744|https://issues.apache.org/jira/browse/KAFKA-5744] and marked this
as a duplicate.
> ShellTest.testRunProgramWithErrorReturn fails locally
> -
>
> Key: KAFKA-5753
> URL: https://issues.apache.org/jira/browse/KAFKA-5753
> Project: Kafka
> Issue Type: Bug
>Reporter: Jason Gustafson
>
> Seeing this locally (on Mac OS):
> {code}
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.kafka.common.utils.ShellTest.testRunProgramWithErrorReturn(ShellTest.java:69)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> The failing assertion is this:
> {code}
> assertTrue(e.getMessage().contains("No such file"));
> {code}
> The actual exception is this:
> {code}
> org.apache.kafka.common.utils.Shell$ExitCodeException: head: illegal byte
> count -- 0
> at org.apache.kafka.common.utils.Shell.runCommand(Shell.java:130)
> at org.apache.kafka.common.utils.Shell.run(Shell.java:76)
> at
> org.apache.kafka.common.utils.Shell$ShellCommandExecutor.execute(Shell.java:204)
> at org.apache.kafka.common.utils.Shell.execCommand(Shell.java:268)
> at org.apache.kafka.common.utils.Shell.execCommand(Shell.java:255)
> at
> org.apache.kafka.common.utils.ShellTest.testRunProgramWithErrorReturn(ShellTest.java:66)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KAFKA-5753) ShellTest.testRunProgramWithErrorReturn fails locally
[
https://issues.apache.org/jira/browse/KAFKA-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huxihx resolved KAFKA-5753.
---
Resolution: Duplicate
> ShellTest.testRunProgramWithErrorReturn fails locally
> -
>
> Key: KAFKA-5753
> URL: https://issues.apache.org/jira/browse/KAFKA-5753
> Project: Kafka
> Issue Type: Bug
>Reporter: Jason Gustafson
>
> Seeing this locally (on Mac OS):
> {code}
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.kafka.common.utils.ShellTest.testRunProgramWithErrorReturn(ShellTest.java:69)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> The failing assertion is this:
> {code}
> assertTrue(e.getMessage().contains("No such file"));
> {code}
> The actual exception is this:
> {code}
> org.apache.kafka.common.utils.Shell$ExitCodeException: head: illegal byte
> count -- 0
> at org.apache.kafka.common.utils.Shell.runCommand(Shell.java:130)
> at org.apache.kafka.common.utils.Shell.run(Shell.java:76)
> at
> org.apache.kafka.common.utils.Shell$ShellCommandExecutor.execute(Shell.java:204)
> at org.apache.kafka.common.utils.Shell.execCommand(Shell.java:268)
> at org.apache.kafka.common.utils.Shell.execCommand(Shell.java:255)
> at
> org.apache.kafka.common.utils.ShellTest.testRunProgramWithErrorReturn(ShellTest.java:66)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KAFKA-5742) Support passing ZK chroot in system tests
[ https://issues.apache.org/jira/browse/KAFKA-5742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134700#comment-16134700 ] ASF GitHub Bot commented on KAFKA-5742: --- Github user xvrl closed the pull request at: https://github.com/apache/kafka/pull/3690 > Support passing ZK chroot in system tests > - > > Key: KAFKA-5742 > URL: https://issues.apache.org/jira/browse/KAFKA-5742 > Project: Kafka > Issue Type: Test > Components: system tests >Reporter: Xavier Léauté >Assignee: Xavier Léauté > Fix For: 0.11.0.1, 1.0.0 > > > Currently spinning up multiple Kafka clusters in a system tests requires at > least one ZK node per Kafka cluster, which wastes a lot of resources. We > currently also don't test anything outside of the ZK root path. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5748) Fix console producer to set timestamp and partition
[ https://issues.apache.org/jira/browse/KAFKA-5748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134742#comment-16134742 ] ASF GitHub Bot commented on KAFKA-5748: --- Github user rancp closed the pull request at: https://github.com/apache/kafka/pull/3689 > Fix console producer to set timestamp and partition > --- > > Key: KAFKA-5748 > URL: https://issues.apache.org/jira/browse/KAFKA-5748 > Project: Kafka > Issue Type: Improvement >Reporter: Ran Ma > Fix For: 0.11.0.1 > > > https://github.com/apache/kafka/pull/3689 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5748) Fix console producer to set timestamp and partition
[ https://issues.apache.org/jira/browse/KAFKA-5748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16134746#comment-16134746 ] ASF GitHub Bot commented on KAFKA-5748: --- GitHub user rancp opened a pull request: https://github.com/apache/kafka/pull/3704 KAFKA-5748 Fix console producer to set timestamp and partition You can merge this pull request into a Git repository by running: $ git pull https://github.com/rancp/kafka cproducer Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3704.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3704 commit 0e1036d3fd1ec86a8ed38f529c93fc2462c4970d Author: Ran Ma Date: 2017-08-17T21:29:02Z fix console producer to parse timestamp and partition > Fix console producer to set timestamp and partition > --- > > Key: KAFKA-5748 > URL: https://issues.apache.org/jira/browse/KAFKA-5748 > Project: Kafka > Issue Type: Improvement >Reporter: Ran Ma > Fix For: 0.11.0.1 > > > https://github.com/apache/kafka/pull/3689 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
