[GitHub] kafka pull request #3547: KAFKA-5608: Add --wait option for JmxTool and use ...

GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/3547 KAFKA-5608: Add --wait option for JmxTool and use in system tests to avoid race between JmxTool and monitored services You can merge this pull request into a Git repository by running: $ git

Consumer blocked when auth failed.

Hi, I was integrating Kafka with Spark, using DirectStream, when my authentication fail, the stream just blocked. No log, no exceptions were thrown. Could some one help to address such situtation

Re: [VOTE] KIP-168: Add TotalTopicCount metric per cluster

+1 Thanks for the KIP. On Mon, Jul 17, 2017 at 9:59 AM, Abhishek Mendhekar < abhishek.mendhe...@gmail.com> wrote: > Dong, Onur, Mickael, Vahid, and Joel: > > Thank you all for voting! > > Best, > Abhishek > > On Thu, Jul 13, 2017 at 8:43 PM, Joel Koshy wrote: > > > +1 > >

[jira] [Created] (KAFKA-5609) Connect log4j should log to file by default

Yeva Byzek created KAFKA-5609: - Summary: Connect log4j should log to file by default Key: KAFKA-5609 URL: https://issues.apache.org/jira/browse/KAFKA-5609 Project: Kafka Issue Type: Improvement

Re: [DISCUSS] 2017 October release planning and release version

With regards to the annotations, I think we should expect that we'll always have some @Evolving APIs. Even though much of the platform is mature, we'll continue to improve and extend it. I'm generally not a fan of @Unstable (since there's rarely a reason to make breaking changes in bug fix

Re: [DISCUSS] 2017 October release planning and release version

Currently the only @unstable annotations left are on Streams and one class of Security modules, and I think we have a good chance of removing them all in the next release. We also have a few @evolving annotations on the Admin, Streams, Security modules etc. And I think we can try to also

Re: [DISCUSS] 2017 October release planning and release version

On Tue, Jul 18, 2017 at 3:42 PM, Ismael Juma wrote: > Hi Guozhang, > > Thanks for volunteering to be the release manager for the next release! > > I am +1 on naming the next release 1.0.0. As you said, Kafka is mature > enough and this will make it easier for others to

Re: [DISCUSS] 2017 October release planning and release version

Also fine with the change in general. As you mentioned, 1.x indicates mature APIs, compatibility and stability. Are we going to remove the @unstable annotations in this release? Gwen On Tue, Jul 18, 2017 at 3:43 PM Ismael Juma wrote: > Hi Guozhang, > > Thanks for

Re: [DISCUSS] 2017 October release planning and release version

Hi Guozhang, Thanks for volunteering to be the release manager for the next release! I am +1 on naming the next release 1.0.0. As you said, Kafka is mature enough and this will make it easier for others to understand our versioning policy. A couple of minor questions inline. On Tue, Jul 18,

[DISCUSS] 2017 October release planning and release version

Hi all, With 0.11.0.0 out of the way, I would like to volunteer to be the release manager for our next time-based feature release. See https://cwiki.apache.org/ confluence/display/KAFKA/Time+Based+Release+Plan if you missed previous communication on time-based releases or need a reminder. I put

[GitHub] kafka-site pull request #67: Updated docs nav and docs toc

Github user asfgit closed the pull request at: https://github.com/apache/kafka-site/pull/67 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] kafka-site issue #67: Updated docs nav and docs toc

Github user guozhangwang commented on the issue: https://github.com/apache/kafka-site/pull/67 Merged to `asf-site`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] kafka-site issue #67: Updated docs nav and docs toc

Github user guozhangwang commented on the issue: https://github.com/apache/kafka-site/pull/67 LGTM. @derrickdoo could you have a separate PR for the `toc.html` changes in `kafka` repo as well? --- If your project is set up for it, you can reply to this email and have your reply

Re: Consumer offsets partitions size much bigger than others

It's possible that the log-cleaning thread has crashed. That is the thread that implements log compaction. Look in the log-cleaner.log file in your kafka debuglog directory to see if there is any indication that it has crashed (error messages, stack traces, etc). What version of kafka are you

[jira] [Created] (KAFKA-5608) System test failure due to timeout starting Jmx tool

Jason Gustafson created KAFKA-5608: -- Summary: System test failure due to timeout starting Jmx tool Key: KAFKA-5608 URL: https://issues.apache.org/jira/browse/KAFKA-5608 Project: Kafka Issue

[GitHub] kafka pull request #3546: MINOR: Apply extra serialized rsync step to both p...

GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/3546 MINOR: Apply extra serialized rsync step to both parallel and serial paths You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafka

Build failed in Jenkins: kafka-trunk-jdk7 #2531

See Changes: [me] MINOR: Add another common error case for CorruptRecordException's error -- [...truncated 969.20 KB...] kafka.server.epoch.LeaderEpochFileCacheTest >

[GitHub] kafka pull request #3541: MINOR: Add another common error case for CorruptRe...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3541 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [DISCUSS] KIP-175: Additional '--describe' views for ConsumerGroupCommand

Hi Jason, I updated the KIP based on your earlier suggestions: https://cwiki.apache.org/confluence/display/KAFKA/KIP-175%3A+Additional+%27--describe%27+views+for+ConsumerGroupCommand The only thing I am wondering at this point is whether it's worth to have a `--describe --offsets` option that

Jenkins build is back to normal : kafka-trunk-jdk8 #1818

See

Re: [VOTE] KIP-134: Delay initial consumer group rebalance

Becket, Thanks for the replies. Now I see you want to optimize with the heuristic that "if I see a JoinGroup shortly enough after a rebalance is completed, then likely there are more JoinGroups coming". I agree that it will help with a single-instance console consumer for debugging etc, but still

Jenkins build is back to normal : kafka-trunk-jdk7 #2530

See

Build failed in Jenkins: kafka-trunk-jdk7 #2529

See Changes: [ismael] MINOR: Use static imports in KafkaLog4jAppender -- [...truncated 971.97 KB...] kafka.server.ReplicaFetcherThreadFatalErrorTest >

[jira] [Reopened] (KAFKA-5217) Improve Streams internal exception handling

[ https://issues.apache.org/jira/browse/KAFKA-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reopened KAFKA-5217: -- [~enothereska] The parent JIRA is an umbrella story of improving error handling in Streams, while

[jira] [Reopened] (KAFKA-5302) Improve exception handling on streams client (communication with brokers)

[ https://issues.apache.org/jira/browse/KAFKA-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reopened KAFKA-5302: -- [~enothereska] Let's leave this ticket open until we have done the migration; also we can use this

[jira] [Reopened] (KAFKA-5301) Improve exception handling on consumer path

[ https://issues.apache.org/jira/browse/KAFKA-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reopened KAFKA-5301: -- [~enothereska] I think we should not resolve this JIRA in a hurry. Have you made a thorough pass

[GitHub] kafka pull request #3241: KAFKA-5384: Enable topic deletion by default

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3241 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Resolved] (KAFKA-5384) KIP-162: Enable topic deletion by default

[ https://issues.apache.org/jira/browse/KAFKA-5384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gwen Shapira resolved KAFKA-5384. - Resolution: Fixed Fix Version/s: (was: 0.12.0.0) 0.11.1.0 Issue

Reason why TopicFilter and related classes are deprecated

Hi devs, why the TopicFilter and related Whitelist and Blacklist classes are marked as deprecated ? They are used for filtering on topic with a regex in the ConsoleConsumer, MirrorMaker, TopicCommand ... Do you have a plan for them to be replaced or not, so marking them as deprecated ?

[GitHub] kafka pull request #3349: MINOR: Used Kafka native constants in KafkaLog4jAp...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3349 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Consumer offsets partitions size much bigger than others

Hi One of our Kafka brokers was running out of disk space and when we checked the file size in the kafka log dir we observed the following $ du -h . --max-depth=2 | grep '__consumer_offsets' 4.0K./kafka-logs/__consumer_offsets-16 4.0K./kafka-logs/__consumer_offsets-40 35G

[jira] [Resolved] (KAFKA-5301) Improve exception handling on consumer path

[ https://issues.apache.org/jira/browse/KAFKA-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska resolved KAFKA-5301. - Resolution: Won't Fix > Improve exception handling on consumer path >

[GitHub] kafka pull request #3545: KAFKA-5499: Minor cleanup of commit code

GitHub user enothereska opened a pull request: https://github.com/apache/kafka/pull/3545 KAFKA-5499: Minor cleanup of commit code You can merge this pull request into a Git repository by running: $ git pull https://github.com/enothereska/kafka KAFKA-5499-commit-exceptions

Re: [DISCUSS] KIP-176: Remove deprecated new-consumer option in ConsoleConsumer tool

On Mon, Jul 17, 2017 at 10:11 AM, Matthias J. Sax wrote: > I thinks, for backward compatibility, this check should be there. The > only change should be, that if `--bootstrap-servers` is used, one can > omit `--new-consumer` (and if used, as warning is printed). > You can

Re: [DISCUSS] KIP-176: Remove deprecated new-consumer option in ConsoleConsumer tool

Hi Matthias, the KIP-176 is for removing so not having --new-consumer anymore. What you are speaking about, I guess, is the deprecated step that isn't addressed by the KIP. I already have a PR for deprecation https://github.com/apache/kafka/pull/3537 which is addressed in the way you are

[jira] [Resolved] (KAFKA-4669) KafkaProducer.flush hangs when NetworkClient.handleCompletedReceives throws exception

[ https://issues.apache.org/jira/browse/KAFKA-4669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-4669. Resolution: Fixed Assignee: Rajini Sivaram Closing this for now. If it reoccurs, let's

Build failed in Jenkins: kafka-trunk-jdk8 #1817

See -- [...truncated 2.46 MB...] org.apache.kafka.streams.integration.KStreamsFineGrainedAutoResetIntegrationTest > shouldThrowStreamsExceptionNoResetSpecified STARTED

[GitHub] kafka pull request #3544: KAFKA-5549 : Explain that 'client.id' is just used...

GitHub user PranavManiar opened a pull request: https://github.com/apache/kafka/pull/3544 KAFKA-5549 : Explain that 'client.id' is just used as a prefix within Streams - Added new String CLIENT_ID_DOC in StreamsConfig for explanation You can merge this pull request into a Git

[jira] [Resolved] (KAFKA-5300) Improve exception handling on producer path

[ https://issues.apache.org/jira/browse/KAFKA-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska resolved KAFKA-5300. - Resolution: Duplicate Duplicate of KAFKA-5006 > Improve exception handling on producer path >

[jira] [Resolved] (KAFKA-5302) Improve exception handling on streams client (communication with brokers)

[ https://issues.apache.org/jira/browse/KAFKA-5302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska resolved KAFKA-5302. - Resolution: Won't Fix > Improve exception handling on streams client (communication with brokers)

[jira] [Resolved] (KAFKA-5217) Improve Streams internal exception handling

[ https://issues.apache.org/jira/browse/KAFKA-5217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eno Thereska resolved KAFKA-5217. - Resolution: Duplicate > Improve Streams internal exception handling >

Jenkins build is back to normal : kafka-trunk-jdk8 #1816

See

[GitHub] kafka pull request #3490: MINOR: Pass over DSL documentation up to stateful ...

Github user enothereska closed the pull request at: https://github.com/apache/kafka/pull/3490 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Jenkins build is back to normal : kafka-trunk-jdk7 #2527

See

[GitHub] kafka pull request #3543: MINOR: Remove flaky assertion in verifyCloseOldest...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3543 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[jira] [Created] (KAFKA-5607) Improve error handling in SocketServer to reduce illegal states later

Rajini Sivaram created KAFKA-5607: - Summary: Improve error handling in SocketServer to reduce illegal states later Key: KAFKA-5607 URL: https://issues.apache.org/jira/browse/KAFKA-5607 Project: Kafka

[GitHub] kafka-site issue #67: Updated docs nav and docs toc

Github user miguno commented on the issue: https://github.com/apache/kafka-site/pull/67 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Created] (KAFKA-5606) Review consumer's RequestFuture usage pattern

Ismael Juma created KAFKA-5606: -- Summary: Review consumer's RequestFuture usage pattern Key: KAFKA-5606 URL: https://issues.apache.org/jira/browse/KAFKA-5606 Project: Kafka Issue Type: Bug

[GitHub] kafka pull request #3543: MINOR: Remove flaky assertion in verifyCloseOldest...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/3543 MINOR: Remove flaky assertion in verifyCloseOldestConnectionWithStagedReceives This seems to fail a lot in Jenkins although it always passes locally for me. Removing the assertion to restore

Build failed in Jenkins: kafka-0.11.0-jdk7 #228

See -- [...truncated 2.07 MB...] org.apache.kafka.clients.producer.MockProducerTest > testAutoCompleteMock PASSED org.apache.kafka.clients.producer.MockProducerTest >

Build failed in Jenkins: kafka-trunk-jdk7 #2526

See -- [...truncated 2.08 MB...] org.apache.kafka.clients.producer.KafkaProducerTest > testInvalidSocketSendBufferSize PASSED org.apache.kafka.clients.producer.KafkaProducerTest >