[GitHub] kafka pull request #1977: HOTFIX: Cannot Stop Kafka with Shell Script (Solut...

GitHub user Mabin-J opened a pull request: https://github.com/apache/kafka/pull/1977 HOTFIX: Cannot Stop Kafka with Shell Script (Solution 2) If Kafka's homepath is long, kafka cannot stop with 'kafka-server-stop.sh'. That command showed this message: ``` No kafka se

[GitHub] kafka pull request #1976: HOTFIX: Cannot Stop Kafka with Shell Script (Solut...

GitHub user Mabin-J opened a pull request: https://github.com/apache/kafka/pull/1976 HOTFIX: Cannot Stop Kafka with Shell Script (Solution 1) If Kafka's homepath is long, kafka cannot stop with 'kafka-server-stop.sh'. That command showed this message: ``` No kafka se

[GitHub] kafka-site pull request #21: Theme enhancement

Github user derrickdoo closed the pull request at: https://github.com/apache/kafka-site/pull/21 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] kafka pull request #1975: HOTFIX: Cannot Stop with 'kafka-server-stop.sh'

Github user Mabin-J closed the pull request at: https://github.com/apache/kafka/pull/1975 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[jira] [Resolved] (KAFKA-3374) Failure in security rolling upgrade phase 2 system test

[ https://issues.apache.org/jira/browse/KAFKA-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3374. Resolution: Fixed Assignee: Flavio Junqueira (was: Ben Stopford) Fix Version/s:

[GitHub] kafka pull request #1975: HOTFIX: Cannot Stop with 'kafka-server-stop.sh'

GitHub user Mabin-J opened a pull request: https://github.com/apache/kafka/pull/1975 HOTFIX: Cannot Stop with 'kafka-server-stop.sh' If Kafka's homepath is long, kafka cannot stop with 'kafka-server-stop.sh'. That command showed this message: ``` No kafka server to s

Re: [DISCUSS] KIP-82 - Add Record Headers

@Kostya Regarding "To get around this we have an awful *cough* solution whereby we have to send our message wrapper with the headers and null content, and then we have an application that has to consume from all the compacted topics and when it sees this message it produces back in a null payload

Build failed in Jenkins: kafka-0.10.1-jdk7 #48

See Changes: [ismael] KAFKA-3985; Transient system test failure -- [...truncated 5372 lines...] kafka.api.UserQuotaTest > testProducerConsumerOverrideUnthrottled STARTED kafka.api.UserQuotaTest

Re: [DISCUSS] KIP-82 - Add Record Headers

@Nacho > > - Brokers can't see the headers (part of the "V" black box)> > > (Also, it would be nice if we had a way to access the headers from the > > brokers, something that is not trivial at this time with the current > broker > > architecture). > > I think this can be addressed with broker i

Build failed in Jenkins: kafka-trunk-jdk8 #949

See Changes: [ismael] KAFKA-3985; Transient system test failure -- [...truncated 7700 lines...] kafka.api.ProducerFailureHandlingTest > testResponseTooLargeForReplicationWithAckAll STARTED kafk

Jenkins build is back to normal : kafka-trunk-jdk7 #1604

See

Re: [DISCUSS] KIP-82 - Add Record Headers

Thank you Radai, Ints are good, I was just asking more of curiosity to understand the reasoning in the KIP. In your reply to Gwen you gave some insight into LinkedIn's internal solution. We do the same, and have a custom Avro wrapper also, but suffer with the same issues that are detailed in

Re: [DISCUSS] KIP-82 - Add Record Headers

@Kostya - we do not support compacted topics in combination with in-house headers. internal systems that use compacted topics (and need to be able to send null values) use the open source producer and cannot use some of the company infra. yet another good use case for "official" header support :-)

Re: [DISCUSS] KIP-82 - Add Record Headers

+1 This is a much needed feature, one I also have been waiting for, and that Kafka has been too long without. Especially since compaction the custom wrapper solution does not work where you want a null payload but need the record to have headers. (It actually made me sign up to the mailing

Re: [DISCUSS] KIP-82 - Add Record Headers

There are pros and cons to having the headers be completely layered above the broker (and hence wire level messages), let's look at the base ones. A few pros (of headers being a higher layer thing): - No broker changes - No protocol changes - Messages with headers can work with brokers that don't

Re: Snazzy new look to our website

> On Oct 4, 2016, at 19:38, Guozhang Wang wrote: > > The new look is great, thanks Derrick and Gwen! +1 it’s excellent! > > I'm wondering if we should still consider breaking "document.html" into > multiple pages and indexed as sub-topics on the left bar? +1 absolutely, this should _definite

[GitHub] kafka-site pull request #21: Theme enhancement

GitHub user derrickdoo opened a pull request: https://github.com/apache/kafka-site/pull/21 Theme enhancement - Added Apache branding guideline stuff - 302 redirects for all the odd direct links to partials that are currently floating around - Improved mobile scroll interacti

[jira] [Commented] (KAFKA-3985) Transient system test failure ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol

[ https://issues.apache.org/jira/browse/KAFKA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15550072#comment-15550072 ] ASF GitHub Bot commented on KAFKA-3985: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #1973: KAFKA-3985: Transient system test failure ZooKeepe...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1973 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-3985) Transient system test failure ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol

[ https://issues.apache.org/jira/browse/KAFKA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-3985. Resolution: Fixed Fix Version/s: 0.10.1.0 Issue resolved by pull request 1973 [https://github

Re: [DISCUSS] KIP-82 - Add Record Headers

Linkedin currently just forces everyone using kafka to: 1. use avro as payload 2. add company-wide headers into their avro schema this leads to problems with teams that would like to produce arbitrary blobs while still benefitting from company-wide infra. it also means every change to these field

Re: [DISCUSS] KIP-82 - Add Record Headers

Since LinkedIn has some kind of wrapper thingy that adds the headers, where they could have added them to Apache Kafka - I'm very curious to hear what drove that decision and the pros/cons of managing the headers outside Kafka itself. On Wed, Oct 5, 2016 at 2:16 PM, K Burstev wrote: > +1 > > This

Re: [VOTE] KIP-47 - Add timestamp-based log deletion policy

Bumping for visibility. KIP is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-47+-+Add+timestamp-based+log+deletion+policy On Wed, Aug 24, 2016 at 2:32 PM Bill Warshaw wrote: > Hello Guozhang, > > KIP-71 seems unrelated to this KIP. KIP-47 is just adding a new deletion > policy (m

[GitHub] kafka pull request #1974: AN-75711 Spike for Kafka server side changes to pr...

GitHub user silpamittapalli opened a pull request: https://github.com/apache/kafka/pull/1974 AN-75711 Spike for Kafka server side changes to prevent a broker from being leader You can merge this pull request into a Git repository by running: $ git pull https://github.com/silp

Re: [DISCUSS] Fault injection tests for Kafka

YES! One of my goal for the fault-injection in our system tests is that whoever fixes the issue will also add tests to make sure it stays fixed. On Wed, Oct 5, 2016 at 11:33 AM, Tom Crayford wrote: > I did some stuff like this recently with simple calls to `tc` (samples that > I used were in the

Build failed in Jenkins: kafka-trunk-jdk8 #948

See Changes: [wangguoz] KAFKA-4176: Only call printStream.flush for System.out -- [...truncated 12037 lines...] org.apache.kafka.common.config.ConfigDefTest > testValidateMissingConfigKey STARTE

Build failed in Jenkins: kafka-0.10.1-jdk7 #47

See Changes: [wangguoz] KAFKA-4176: Only call printStream.flush for System.out -- [...truncated 6993 lines...] kafka.log.FileMessageSetTest > testIteratorIsConsistent STARTED kafka.log.FileMessa

Re: [DISCUSS] Fault injection tests for Kafka

I did some stuff like this recently with simple calls to `tc` (samples that I used were in the README for https://github.com/tylertreat/comcast). The only notable bug I found so far is that if you cut all the kafka nodes entirely off from zookeeper for say, 60 seconds, then reconnect them, the node

Re: [DISCUSS] Fault injection tests for Kafka

Yeah, totally agree on discussing what we want to test first and implement anything later :) Its just that whenever I have this discussion Jepsen came up, so I was curious what was driving the interest and whether the specific framework is important to the community. On Tue, Oct 4, 2016 at 5:46 P

[jira] [Commented] (KAFKA-4256) Use IP for ZK broker register

[ https://issues.apache.org/jira/browse/KAFKA-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549507#comment-15549507 ] Jason Gustafson commented on KAFKA-4256: [~yarekt] We have a broker configuration

Build failed in Jenkins: kafka-trunk-jdk7 #1603

See Changes: [wangguoz] KAFKA-4176: Only call printStream.flush for System.out -- [...truncated 3821 lines...] kafka.coordinator.GroupCoordinatorResponseTest > testSyncGroupLeaderAfterFollower

[jira] [Commented] (KAFKA-3985) Transient system test failure ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol

[ https://issues.apache.org/jira/browse/KAFKA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549319#comment-15549319 ] Geoff Anderson commented on KAFKA-3985: --- Yes, using canonical location in /tmp is ab

[jira] [Commented] (KAFKA-4254) Questionable handling of unknown partitions in KafkaProducer

[ https://issues.apache.org/jira/browse/KAFKA-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549298#comment-15549298 ] Ismael Juma commented on KAFKA-4254: Yeah, one refresh makes sense to me. > Questiona

[jira] [Commented] (KAFKA-3985) Transient system test failure ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol

[ https://issues.apache.org/jira/browse/KAFKA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549221#comment-15549221 ] Flavio Junqueira commented on KAFKA-3985: - I've created a pull request for this, f

[jira] [Issue Comment Deleted] (KAFKA-4254) Questionable handling of unknown partitions in KafkaProducer

[ https://issues.apache.org/jira/browse/KAFKA-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-4254: --- Comment: was deleted (was: [~ijuma] I'm sympathetic to your concern, but it seems a little dif

[jira] [Commented] (KAFKA-4254) Questionable handling of unknown partitions in KafkaProducer

[ https://issues.apache.org/jira/browse/KAFKA-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549204#comment-15549204 ] Jason Gustafson commented on KAFKA-4254: [~ijuma] I'm sympathetic to your concern,

[jira] [Commented] (KAFKA-4254) Questionable handling of unknown partitions in KafkaProducer

[ https://issues.apache.org/jira/browse/KAFKA-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549203#comment-15549203 ] Jason Gustafson commented on KAFKA-4254: [~ijuma] I'm sympathetic to your concern,

[jira] [Assigned] (KAFKA-3985) Transient system test failure ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol

[ https://issues.apache.org/jira/browse/KAFKA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira reassigned KAFKA-3985: --- Assignee: Flavio Junqueira > Transient system test failure > ZooKeeperSecurityUpgrad

[jira] [Commented] (KAFKA-3985) Transient system test failure ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol

[ https://issues.apache.org/jira/browse/KAFKA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549175#comment-15549175 ] ASF GitHub Bot commented on KAFKA-3985: --- GitHub user fpj opened a pull request:

[GitHub] kafka pull request #1973: KAFKA-3985: Transient system test failure ZooKeepe...

GitHub user fpj opened a pull request: https://github.com/apache/kafka/pull/1973 KAFKA-3985: Transient system test failure ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol You can merge this pull request into a Git repository by running: $ git pull http

[jira] [Assigned] (KAFKA-4176) Node stopped receiving heartbeat responses once another node started within the same group

[ https://issues.apache.org/jira/browse/KAFKA-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang reassigned KAFKA-4176: Assignee: Guozhang Wang > Node stopped receiving heartbeat responses once another node star

[jira] [Resolved] (KAFKA-4176) Node stopped receiving heartbeat responses once another node started within the same group

[ https://issues.apache.org/jira/browse/KAFKA-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4176. -- Resolution: Fixed Fix Version/s: 0.10.1.1 0.10.2.0 Issue resolved by p

[jira] [Commented] (KAFKA-4176) Node stopped receiving heartbeat responses once another node started within the same group

[ https://issues.apache.org/jira/browse/KAFKA-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15549073#comment-15549073 ] ASF GitHub Bot commented on KAFKA-4176: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #1965: KAFKA-4176: Only call printStream.flush for System...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1965 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Support for Kafka

Dear Kafka team. I am in the Implementation stage of Kafka cluster and looking to find out does Apache Kafka supported for Ubuntu 16.04 LTS - Xenial. Would be great if you please let us know. [The Exchange Lab] Syed Hussaini Infrastructure Engineer 1 Ne

[jira] [Commented] (KAFKA-3224) Add timestamp-based log deletion policy

[ https://issues.apache.org/jira/browse/KAFKA-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548776#comment-15548776 ] ASF GitHub Bot commented on KAFKA-3224: --- GitHub user bill-warshaw opened a pull requ

[GitHub] kafka pull request #1972: KAFKA-3224: New log deletion policy based on times...

GitHub user bill-warshaw opened a pull request: https://github.com/apache/kafka/pull/1972 KAFKA-3224: New log deletion policy based on timestamp * adds a new topic-level broker configuration, `log.retention.min.timestamp` * if unset, this setting is ignored * setting this

[jira] [Commented] (KAFKA-4257) Inconsistencies in 0.10.1 upgrade docs

[ https://issues.apache.org/jira/browse/KAFKA-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548729#comment-15548729 ] Jeff Klukas commented on KAFKA-4257: The pull request definitely helps. Thanks for jum

[jira] [Commented] (KAFKA-4257) Inconsistencies in 0.10.1 upgrade docs

[ https://issues.apache.org/jira/browse/KAFKA-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548714#comment-15548714 ] Ismael Juma commented on KAFKA-4257: [~jeff.klu...@gmail.com], does the following help

[GitHub] kafka pull request #1971: MINOR: Clarify 0.10.1.0 upgrade docs

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/1971 MINOR: Clarify 0.10.1.0 upgrade docs This is a minor change to fix the most glaring issues. We have another JIRA to revamp the upgrade docs. You can merge this pull request into a Git repository by r

[jira] [Commented] (KAFKA-4246) Discretionary partition assignment on the consumer side not functional

[ https://issues.apache.org/jira/browse/KAFKA-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548700#comment-15548700 ] Alexandru Ionita commented on KAFKA-4246: - It seems the {{unsubscribe()}} call doe

[jira] [Created] (KAFKA-4257) Inconsistencies in 0.10.1 upgrade docs

Jeff Klukas created KAFKA-4257: -- Summary: Inconsistencies in 0.10.1 upgrade docs Key: KAFKA-4257 URL: https://issues.apache.org/jira/browse/KAFKA-4257 Project: Kafka Issue Type: Bug C

[jira] [Created] (KAFKA-4256) Use IP for ZK broker register

Yarek Tyshchenko created KAFKA-4256: --- Summary: Use IP for ZK broker register Key: KAFKA-4256 URL: https://issues.apache.org/jira/browse/KAFKA-4256 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-4253) Fix Kafka Stream thread shutting down process ordering

[ https://issues.apache.org/jira/browse/KAFKA-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548284#comment-15548284 ] ASF GitHub Bot commented on KAFKA-4253: --- GitHub user dguy opened a pull request:

[GitHub] kafka pull request #1970: KAFKA-4253: Fix Kafka Stream thread shutting down ...

GitHub user dguy opened a pull request: https://github.com/apache/kafka/pull/1970 KAFKA-4253: Fix Kafka Stream thread shutting down process ordering Changed the ordering in `StreamThread.shutdown` 1. commitAll (we need to commit so that any cached data is flushed through the top

[jira] [Commented] (KAFKA-789) Producer-side persistence for delivery guarantee

[ https://issues.apache.org/jira/browse/KAFKA-789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548191#comment-15548191 ] Ismael Juma commented on KAFKA-789: --- We are tracking this issue via https://issues.apache

[jira] [Commented] (KAFKA-156) Messages should not be dropped when brokers are unavailable

[ https://issues.apache.org/jira/browse/KAFKA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548193#comment-15548193 ] Ismael Juma commented on KAFKA-156: --- To clarify, the issue where are tracking this is now

[jira] [Updated] (KAFKA-156) Messages should not be dropped when brokers are unavailable

[ https://issues.apache.org/jira/browse/KAFKA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-156: -- Fix Version/s: (was: 0.10.1.0) > Messages should not be dropped when brokers are unavailable > --

[jira] [Commented] (KAFKA-3985) Transient system test failure ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol

[ https://issues.apache.org/jira/browse/KAFKA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548160#comment-15548160 ] Rajini Sivaram commented on KAFKA-3985: --- [~fpj] At the moment they use a fixed filen

[jira] [Comment Edited] (KAFKA-3985) Transient system test failure ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol

[ https://issues.apache.org/jira/browse/KAFKA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548125#comment-15548125 ] Flavio Junqueira edited comment on KAFKA-3985 at 10/5/16 9:02 AM: --

[jira] [Commented] (KAFKA-3985) Transient system test failure ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol

[ https://issues.apache.org/jira/browse/KAFKA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548125#comment-15548125 ] Flavio Junqueira commented on KAFKA-3985: - [~rsivaram] .bq I wouldn't have expect

[jira] [Commented] (KAFKA-3985) Transient system test failure ZooKeeperSecurityUpgradeTest.test_zk_security_upgrade.security_protocol

[ https://issues.apache.org/jira/browse/KAFKA-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15548091#comment-15548091 ] Rajini Sivaram commented on KAFKA-3985: --- [~fpj] I would not have expected each test

[jira] [Assigned] (KAFKA-4253) Fix Kafka Stream thread shutting down process ordering

[ https://issues.apache.org/jira/browse/KAFKA-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damian Guy reassigned KAFKA-4253: - Assignee: Damian Guy > Fix Kafka Stream thread shutting down process ordering > -

[jira] [Work started] (KAFKA-4253) Fix Kafka Stream thread shutting down process ordering

[ https://issues.apache.org/jira/browse/KAFKA-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4253 started by Damian Guy. - > Fix Kafka Stream thread shutting down process ordering >

[jira] [Created] (KAFKA-4255) Scheme not compatible with

Alex created KAFKA-4255: --- Summary: Scheme not compatible with Key: KAFKA-4255 URL: https://issues.apache.org/jira/browse/KAFKA-4255 Project: Kafka Issue Type: Bug Reporter: Alex Schemes