[GitHub] kafka pull request: MINOR: Move connect.start() to try catch block

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1347 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3673) Connect tests dont handle concurrent config changes

[ https://issues.apache.org/jira/browse/KAFKA-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276019#comment-15276019 ] ASF GitHub Bot commented on KAFKA-3673: --- Github user asfgit closed the pull request

[jira] [Resolved] (KAFKA-3673) Connect tests dont handle concurrent config changes

[ https://issues.apache.org/jira/browse/KAFKA-3673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-3673. -- Resolution: Fixed Fix Version/s: 0.10.1.0 Issue resolved by pull request

[GitHub] kafka pull request: KAFKA-3673: Connect tests don't handle concurr...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1340 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[DISCUSS] KIP-59 - Proposal for a kafka broker command - kafka-brokers.sh

Hi All, This is to start off a discussion on the above KIP at  https://cwiki.apache.org/confluence/display/KAFKA/KIP-59+-+Proposal+for+a+kafka+broker+command+-+kafka-brokers.sh The proposal is to fill the void of a command line tool/utility that can provide information on the cluster and brokers i

[jira] [Commented] (KAFKA-1489) Global threshold on data retention size

[ https://issues.apache.org/jira/browse/KAFKA-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276004#comment-15276004 ] ASF GitHub Bot commented on KAFKA-1489: --- GitHub user bendrees opened a pull request:

[GitHub] kafka pull request: KAFKA-1489: Global threshold on data retention...

GitHub user bendrees opened a pull request: https://github.com/apache/kafka/pull/1348 KAFKA-1489: Global threshold on data retention size Implemented a "log retention policy" based on keeping a certain percentage of disk space free. In dynamic situations where topics are adde

[jira] [Commented] (KAFKA-3647) Unable to set a ssl provider

[ https://issues.apache.org/jira/browse/KAFKA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275995#comment-15275995 ] Johan Abbors commented on KAFKA-3647: - The -keyalg RSA fix Elvar found should be inclu

[jira] [Commented] (KAFKA-3647) Unable to set a ssl provider

[ https://issues.apache.org/jira/browse/KAFKA-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275992#comment-15275992 ] Johan Abbors commented on KAFKA-3647: - Thanks Elvar. Re-generating the keystores with

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275974#comment-15275974 ] Jiangjie Qin commented on KAFKA-3565: - Actually, never mind. The buffer size is not th

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275971#comment-15275971 ] Jiangjie Qin commented on KAFKA-3565: - The batch size was 80K. It is not quite clear t

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275970#comment-15275970 ] Jiangjie Qin commented on KAFKA-3565: - [~junrao] Yes, the value bound was supposed to

[jira] [Resolved] (KAFKA-3671) Topics should not be in common ConnectorConfig definitions

[ https://issues.apache.org/jira/browse/KAFKA-3671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewen Cheslack-Postava resolved KAFKA-3671. -- Resolution: Fixed Fix Version/s: 0.10.1.0 Fixed by https://github.com/apa

[GitHub] kafka pull request: KAFKA 3671: Move topics to SinkConnectorConfig

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1335 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275959#comment-15275959 ] Jun Rao commented on KAFKA-3565: [~becket_qin], also, it seems that you patched ProducerPe

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275957#comment-15275957 ] Jun Rao commented on KAFKA-3565: [~becket_qin], thanks for confirming this. I guess defaul

[GitHub] kafka pull request: MINOR: Move connect.start() to try catch block

GitHub user Ishiihara opened a pull request: https://github.com/apache/kafka/pull/1347 MINOR: Move connect.start() to try catch block You can merge this pull request into a Git repository by running: $ git pull https://github.com/Ishiihara/kafka connect-standalone Alternative

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275948#comment-15275948 ] Jiangjie Qin commented on KAFKA-3565: - BTW, I just created KAFKA-3677 to provide a too

[jira] [Created] (KAFKA-3677) Add a producer performance tuning tool

Jiangjie Qin created KAFKA-3677: --- Summary: Add a producer performance tuning tool Key: KAFKA-3677 URL: https://issues.apache.org/jira/browse/KAFKA-3677 Project: Kafka Issue Type: Improvement

[GitHub] kafka pull request: MINOR: Add virtual env to Kafka system test RE...

GitHub user Ishiihara opened a pull request: https://github.com/apache/kafka/pull/1346 MINOR: Add virtual env to Kafka system test README.md You can merge this pull request into a Git repository by running: $ git pull https://github.com/Ishiihara/kafka add-venv Alternatively

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275934#comment-15275934 ] Jiangjie Qin commented on KAFKA-3565: - [~ijuma] We should absolutely mention the 8 byt

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275930#comment-15275930 ] Guozhang Wang commented on KAFKA-3565: -- Just curious what is the {{batch.size}} in yo

[jira] [Commented] (KAFKA-3665) Default ssl.endpoint.identification.algorithm should be https

[ https://issues.apache.org/jira/browse/KAFKA-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275926#comment-15275926 ] Jun Rao commented on KAFKA-3665: Interesting, the difference is that in https, if a VIP is

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275924#comment-15275924 ] Jiangjie Qin commented on KAFKA-3565: - [~junrao] Yes, you are right. I changed the pro

[jira] [Commented] (KAFKA-1981) Make log compaction point configurable

[ https://issues.apache.org/jira/browse/KAFKA-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275920#comment-15275920 ] Eric Wasserman commented on KAFKA-1981: --- That did it. Thanks. I created https://cwi

[GitHub] kafka pull request: KAFKA-3676: system tests for connector pause/r...

GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/1345 KAFKA-3676: system tests for connector pause/resume You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka KAFKA-3676 Alternatively

[jira] [Commented] (KAFKA-3676) Add system tests for connector pause/resume

[ https://issues.apache.org/jira/browse/KAFKA-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275904#comment-15275904 ] ASF GitHub Bot commented on KAFKA-3676: --- GitHub user hachikuji opened a pull request

[jira] [Commented] (KAFKA-3585) Shutdown slow when there is only one broker which is controller

[ https://issues.apache.org/jira/browse/KAFKA-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275879#comment-15275879 ] Pengwei commented on KAFKA-3585: do you used the kafka-server-stop.sh to shutdown? > Shut

[jira] [Issue Comment Deleted] (KAFKA-3585) Shutdown slow when there is only one broker which is controller

[ https://issues.apache.org/jira/browse/KAFKA-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengwei updated KAFKA-3585: --- Comment: was deleted (was: do you used the kafka-server-stop.sh to shutdown?) > Shutdown slow when there is o

[jira] [Commented] (KAFKA-3585) Shutdown slow when there is only one broker which is controller

[ https://issues.apache.org/jira/browse/KAFKA-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275880#comment-15275880 ] Pengwei commented on KAFKA-3585: do you used the kafka-server-stop.sh to shutdown? > Shut

[jira] [Commented] (KAFKA-3665) Default ssl.endpoint.identification.algorithm should be https

[ https://issues.apache.org/jira/browse/KAFKA-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275873#comment-15275873 ] Ismael Juma commented on KAFKA-3665: [~junrao], good questions. With regards to 1, bot

[GitHub] kafka pull request: Fixup KAFKA-3160: catch decompression errors i...

GitHub user dpkp opened a pull request: https://github.com/apache/kafka/pull/1344 Fixup KAFKA-3160: catch decompression errors in constructor After testing KAFKA-3160 a bit more, I found that the error code was not being set properly in ProduceResponse. This happened because the val

[jira] [Commented] (KAFKA-3160) Kafka LZ4 framing code miscalculates header checksum

[ https://issues.apache.org/jira/browse/KAFKA-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275852#comment-15275852 ] ASF GitHub Bot commented on KAFKA-3160: --- GitHub user dpkp opened a pull request:

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275847#comment-15275847 ] Ismael Juma commented on KAFKA-3565: [~becket_qin], would you be able to submit a PR t

[jira] [Updated] (KAFKA-3674) Connector target state changes not propagated to all workers

[ https://issues.apache.org/jira/browse/KAFKA-3674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3674: --- Fix Version/s: 0.10.0.0 > Connector target state changes not propagated to all workers > -

Re: reading the consumer offsets topic

It looks like you’re just missing the proper message formatter. Of course, that largely depends on your version of the broker. Try: ./kafka-console-consumer.sh --broker-list localhost:9092 --topic __consumer_offsets --formatter kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter If f

Build failed in Jenkins: kafka-0.10.0-jdk7 #65

See Changes: [ismael] KAFKA-3579; Reference both old and new consumer properties in -- [...truncated 5554 lines...] org.apache.kafka.streams.StreamsConfigTest > testGetProducerConfigs PASSED or

Build failed in Jenkins: kafka-trunk-jdk8 #602

See Changes: [ismael] KAFKA-3579; Reference both old and new consumer properties in -- [...truncated 3310 lines...] kafka.log.LogSegmentTest > testRecoveryWithCorruptMessage PASSED kafka.log.Lo

Build failed in Jenkins: kafka-trunk-jdk7 #1266

See Changes: [ismael] KAFKA-3579; Reference both old and new consumer properties in -- [...truncated 6412 lines...] org.apache.kafka.connect.storage.KafkaOffsetBackingStoreTest > testReloadOnS

[jira] [Created] (KAFKA-3676) Add system tests for connector pause/resume

Jason Gustafson created KAFKA-3676: -- Summary: Add system tests for connector pause/resume Key: KAFKA-3676 URL: https://issues.apache.org/jira/browse/KAFKA-3676 Project: Kafka Issue Type: Tes

[jira] [Commented] (KAFKA-3565) Producer's throughput lower with compressed data after KIP-31/32

[ https://issues.apache.org/jira/browse/KAFKA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275793#comment-15275793 ] Jun Rao commented on KAFKA-3565: [~becket_qin], thanks for the latest analysis. The differ

[jira] [Updated] (KAFKA-3587) LogCleaner fails due to incorrect offset map computation on a replica

[ https://issues.apache.org/jira/browse/KAFKA-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3587: --- Reviewer: Jun Rao > LogCleaner fails due to incorrect offset map computation on a replica > --

[jira] [Commented] (KAFKA-3579) TopicCommand references outdated consumer property fetch.message.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275764#comment-15275764 ] Ismael Juma commented on KAFKA-3579: Noting here that the command now references both

[jira] [Updated] (KAFKA-3579) TopicCommand references outdated consumer property fetch.message.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3579: --- Resolution: Fixed Fix Version/s: 0.10.0.0 Status: Resolved (was: Patch Available) I

[jira] [Commented] (KAFKA-3579) TopicCommand references outdated consumer property fetch.message.max.bytes

[ https://issues.apache.org/jira/browse/KAFKA-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275761#comment-15275761 ] ASF GitHub Bot commented on KAFKA-3579: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: KAFKA-3579 - Update reference to the outdated ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1239 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Commented] (KAFKA-3587) LogCleaner fails due to incorrect offset map computation on a replica

[ https://issues.apache.org/jira/browse/KAFKA-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275741#comment-15275741 ] Jun Rao commented on KAFKA-3587: [~guozhang], I don't think (a:4) will ever be duplicated

reading the consumer offsets topic

I'm having difficulty reading the consumer offsets topic from the command line. I try the following but it doesn't seem to work (along with a few related variants including specifying the zookeeper hosts): ./kafka-console-consumer.sh --broker-list localhost:9092 --topic __consumer_offsets Is the

Build failed in Jenkins: kafka-0.10.0-jdk7 #64

See Changes: [junrao] KAFKA-3670; ControlledShutdownLeaderSelector should pick the preferred -- [...truncated 6410 lines...] org.apache.kafka.connect.storage.KafkaOffsetBackingStoreTest > testSe

Build failed in Jenkins: kafka-trunk-jdk8 #601

See Changes: [junrao] KAFKA-3670; ControlledShutdownLeaderSelector should pick the preferred -- [...truncated 5594 lines...] org.apache.kafka.streams.processor.internals.assignment.TaskAssignorT

Build failed in Jenkins: kafka-trunk-jdk7 #1265

See Changes: [junrao] KAFKA-3670; ControlledShutdownLeaderSelector should pick the preferred -- Started by an SCM change [EnvInject] - Loading node environment variables. Building remotely on H1

[jira] [Commented] (KAFKA-3587) LogCleaner fails due to incorrect offset map computation on a replica

[ https://issues.apache.org/jira/browse/KAFKA-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275650#comment-15275650 ] Guozhang Wang commented on KAFKA-3587: -- If I understand correctly, let's day the prod

[jira] [Commented] (KAFKA-3670) ControlledShutdownLeaderSelector should pick the preferred replica as the new leader, if possible

[ https://issues.apache.org/jira/browse/KAFKA-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275638#comment-15275638 ] ASF GitHub Bot commented on KAFKA-3670: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-3670) ControlledShutdownLeaderSelector should pick the preferred replica as the new leader, if possible

[ https://issues.apache.org/jira/browse/KAFKA-3670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-3670: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request 1338 [https:

[GitHub] kafka pull request: KAFKA-3670; ControlledShutdownLeaderSelector s...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1338 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: [VOTE] KIP-45: Standardize KafkaConsumer API to use Collection

Hi Harsha, See inline. On Sun, May 8, 2016 at 3:02 AM, Harsha wrote: > Ismael, >Do we need to add old assign and subscribe that accepts List. Yes, to maintain binary compatibility. If you compile your jar with kafka-clients 0.9.0.1 and then run it with kafka-clients 0.10.0.0 you will

[jira] [Updated] (KAFKA-3675) Add lz4 to parametrized `test_upgrade` system test

[ https://issues.apache.org/jira/browse/KAFKA-3675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-3675: --- Status: Patch Available (was: In Progress) > Add lz4 to parametrized `test_upgrade` system test > ---

[jira] [Commented] (KAFKA-3675) Add lz4 to parametrized `test_upgrade` system test

[ https://issues.apache.org/jira/browse/KAFKA-3675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275506#comment-15275506 ] ASF GitHub Bot commented on KAFKA-3675: --- GitHub user ijuma opened a pull request:

[GitHub] kafka pull request: KAFKA-3675; Add lz4 to parametrized `test_upgr...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/1343 KAFKA-3675; Add lz4 to parametrized `test_upgrade` system test You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka kafka-3675-lz4-test-upg