Should the KafkaProducer callback get the record batch?

Hi, Currently the callback of Producer gets either Metadata (partition + offset) or Exception. In case of an exception, it doesn't get any information other than the exception. Which means that if I want to write failures to a "dead letter queue" of some kind, I have to wait on the Future. I'd li

Build failed in Jenkins: Kafka-trunk #656

See Changes: [wangguoz] KAFKA-2591: Fix StreamingMetrics -- [...truncated 4001 lines...] kafka.coordinator.CoordinatorMetadataTest > testUnbindNonexistentGroupFromTopics PASSED kafka.coordinator.Coo

Re: KAFKA-2364 migrate docs from SVN to git

Thanks Gwen, i will update the next steps. On Oct 3, 2015 1:08 AM, "Gwen Shapira" wrote: > Hi, > > I created asf-git under https://git-wip-us.apache.org/repos/asf/kafka-site > . > git and pushed our existing docs in there. > What do we need to do to get infra to show this in our website? > > Nex

[jira] [Commented] (KAFKA-2591) Remove Persistent Data before Restoringafter a Fault

[ https://issues.apache.org/jira/browse/KAFKA-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942076#comment-14942076 ] ASF GitHub Bot commented on KAFKA-2591: --- Github user asfgit closed the pull request

[jira] [Updated] (KAFKA-2591) Remove Persistent Data before Restoringafter a Fault

[ https://issues.apache.org/jira/browse/KAFKA-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2591: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[GitHub] kafka pull request: KAFKA-2591: Fix StreamingMetrics

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/265 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Commented] (KAFKA-2566) Improve Jenkins set-up

[ https://issues.apache.org/jira/browse/KAFKA-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942071#comment-14942071 ] Guozhang Wang commented on KAFKA-2566: -- [~junrao] Yeah, we plan to run testAll for tr

[jira] [Resolved] (KAFKA-2606) Remove kafka.utils.Time in favour of o.a.kafka.common.utils.Time

[ https://issues.apache.org/jira/browse/KAFKA-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar resolved KAFKA-2606. Resolution: Duplicate Duplicate of: https://issues.apache.org/jira/browse/KAFKA-2247 Ismael

[jira] [Updated] (KAFKA-2247) Merge kafka.utils.Time and kafka.common.utils.Time

[ https://issues.apache.org/jira/browse/KAFKA-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aditya Auradkar updated KAFKA-2247: --- Description: We currently have 2 different versions of Time in clients and core. These need t

[jira] [Commented] (KAFKA-2566) Improve Jenkins set-up

[ https://issues.apache.org/jira/browse/KAFKA-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942034#comment-14942034 ] Jun Rao commented on KAFKA-2566: The trunk Jenkins job actually runs test with both scala

[jira] [Commented] (KAFKA-2477) Replicas spuriously deleting all segments in partition

[ https://issues.apache.org/jira/browse/KAFKA-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941999#comment-14941999 ] Jun Rao commented on KAFKA-2477: [~hakon], thanks a lot for the update. This seems like a

[jira] [Commented] (KAFKA-2606) Remove kafka.utils.Time in favour of o.a.kafka.common.utils.Time

[ https://issues.apache.org/jira/browse/KAFKA-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941998#comment-14941998 ] Ismael Juma commented on KAFKA-2606: Sorry, my bad, I couldn't find it when I searched

[jira] [Commented] (KAFKA-2606) Remove kafka.utils.Time in favour of o.a.kafka.common.utils.Time

[ https://issues.apache.org/jira/browse/KAFKA-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941984#comment-14941984 ] Aditya Auradkar commented on KAFKA-2606: [~ijuma] This is a duplicate of https://i

Build failed in Jenkins: Kafka-trunk #655

See Changes: [wangguoz] KAFKA-2597: Add to .gitignore the Eclipse IDE directories -- [...truncated 844 lines...] kafka.coordinator.ConsumerCoordinatorResponseTest > testJoinGroupInconsistentPartition

[jira] [Commented] (KAFKA-2566) Improve Jenkins set-up

[ https://issues.apache.org/jira/browse/KAFKA-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941939#comment-14941939 ] Ismael Juma commented on KAFKA-2566: Not yet, I can do it tomorrow. > Improve Jenkins

[jira] [Commented] (KAFKA-2566) Improve Jenkins set-up

[ https://issues.apache.org/jira/browse/KAFKA-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941931#comment-14941931 ] Guozhang Wang commented on KAFKA-2566: -- Cool, is there an existing ticket for upgradi

[jira] [Commented] (KAFKA-2566) Improve Jenkins set-up

[ https://issues.apache.org/jira/browse/KAFKA-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941928#comment-14941928 ] Ismael Juma commented on KAFKA-2566: It's the same reason as I just said. We build by

[jira] [Updated] (KAFKA-2597) Add Eclipse directories to .gitignore

[ https://issues.apache.org/jira/browse/KAFKA-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-2597: - Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request

[jira] [Commented] (KAFKA-2597) Add Eclipse directories to .gitignore

[ https://issues.apache.org/jira/browse/KAFKA-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941927#comment-14941927 ] ASF GitHub Bot commented on KAFKA-2597: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: KAFKA-2597: Add to .gitignore the Eclipse IDE ...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/259 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Commented] (KAFKA-2566) Improve Jenkins set-up

[ https://issues.apache.org/jira/browse/KAFKA-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941923#comment-14941923 ] Guozhang Wang commented on KAFKA-2566: -- Agree, will change accordingly. BTW, current

[jira] [Commented] (KAFKA-2580) Kafka Broker keeps file handles open for all log files (even if its not written to/read from)

[ https://issues.apache.org/jira/browse/KAFKA-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941910#comment-14941910 ] Vinoth Chandar commented on KAFKA-2580: --- A LRU file handle is something very commonl

[jira] [Commented] (KAFKA-2566) Improve Jenkins set-up

[ https://issues.apache.org/jira/browse/KAFKA-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941903#comment-14941903 ] Ismael Juma commented on KAFKA-2566: Guozhang, it looks good to me apart from the PR j

Re: KAFKA-2364 migrate docs from SVN to git

On Fri, Oct 2, 2015 at 9:36 PM, Ashish wrote: > > Does this mean doc patches would be welcome only for a production > release? else you would need a diff branch for accepting docs for > trunk features/patches which don't get reflected on the website. > Yes, you will contribute changes to the rele

Re: KAFKA-2364 migrate docs from SVN to git

Thanks Gwen. Comments below. On Fri, Oct 2, 2015 at 8:38 PM, Gwen Shapira wrote: > I created asf-git under https://git-wip-us.apache.org/repos/asf/kafka-site > . > git and pushed our existing docs in there. > What do we need to do to get infra to show this in our website? > You need to reopen t

Build failed in Jenkins: Kafka-trunk #654

See Changes: [junrao] KAFKA-2517; Performance Regression post SSL implementation (zero copy) [wangguoz] MINOR: Fix Vagrant setup script for use on Fedora [wangguoz] MINOR: remove no longer needed CommitType ---

[GitHub] kafka pull request: MINOR: remove no longer needed CommitType

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/258 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] kafka pull request: MINOR: Fix Vagrant setup script for use on Fed...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/246 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Commented] (KAFKA-2517) Performance Regression post SSL implementation

[ https://issues.apache.org/jira/browse/KAFKA-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941820#comment-14941820 ] ASF GitHub Bot commented on KAFKA-2517: --- Github user asfgit closed the pull request

[GitHub] kafka pull request: KAFKA-2517; Performance Regression post SSL im...

Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/273 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[jira] [Updated] (KAFKA-2517) Performance Regression post SSL implementation

[ https://issues.apache.org/jira/browse/KAFKA-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2517: --- Resolution: Fixed Status: Resolved (was: Patch Available) Issue resolved by pull request 273 [https:/

Re: KAFKA-2364 migrate docs from SVN to git

On Fri, Oct 2, 2015 at 12:38 PM, Gwen Shapira wrote: > Hi, > > I created asf-git under https://git-wip-us.apache.org/repos/asf/kafka-site. > git and pushed our existing docs in there. > What do we need to do to get infra to show this in our website? > > Next steps: > 1) Minor fix to PR 171 > 2) Me

Re: KAFKA-2364 migrate docs from SVN to git

Hi, I created asf-git under https://git-wip-us.apache.org/repos/asf/kafka-site. git and pushed our existing docs in there. What do we need to do to get infra to show this in our website? Next steps: 1) Minor fix to PR 171 2) Merge PR 171 3) Get Apache to show our git site 4) Update wiki with "con

Corrupted Message Issue

Hi, I am receiving this in the server logs. [2015-10-02 17:58:31,726] ERROR [KafkaApi-0] Error processing ProducerRequest with correlation id 103815 from client prod-dcs-stats-producer-1034 on partition [prod-dcs-stats,0] (kafka.server.KafkaApis) kafka.message.InvalidMessageException: Message is c

[jira] [Commented] (KAFKA-2452) enable new consumer in mirror maker

[ https://issues.apache.org/jira/browse/KAFKA-2452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941609#comment-14941609 ] Samuel Zhou commented on KAFKA-2452: [~ijuma] thank you for the information.! Another

[jira] [Commented] (KAFKA-2566) Improve Jenkins set-up

[ https://issues.apache.org/jira/browse/KAFKA-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941567#comment-14941567 ] Guozhang Wang commented on KAFKA-2566: -- [~junrao] [~ijuma] [~fpj] I browse through th

[jira] [Commented] (KAFKA-2587) Transient test failure: `SimpleAclAuthorizerTest`

[ https://issues.apache.org/jira/browse/KAFKA-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941533#comment-14941533 ] Parth Brahmbhatt commented on KAFKA-2587: - Hey Ismael, As mentioned above, in pro

[jira] [Commented] (KAFKA-2587) Transient test failure: `SimpleAclAuthorizerTest`

[ https://issues.apache.org/jira/browse/KAFKA-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941502#comment-14941502 ] Ismael Juma commented on KAFKA-2587: Hey Parth, thanks for looking into this. If it's

[jira] [Commented] (KAFKA-2587) Transient test failure: `SimpleAclAuthorizerTest`

[ https://issues.apache.org/jira/browse/KAFKA-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941474#comment-14941474 ] Parth Brahmbhatt commented on KAFKA-2587: - I looked at the code to reason around w

[jira] [Commented] (KAFKA-2566) Improve Jenkins set-up

[ https://issues.apache.org/jira/browse/KAFKA-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941390#comment-14941390 ] Guozhang Wang commented on KAFKA-2566: -- [~fpj] Sure go ahead. > Improve Jenkins set-

Re: Java KafkaClient

Hello Horst, I think you are using Kafka version <= 0.8.2. The new consumer, org.apache.kafka.clients.consumer, is being worked on and will be available in 0.9.0, however it will be marked unstable for 0.9.0. You can follow Apache Kafka docs

[jira] [Updated] (KAFKA-2598) Add Test with authorizer for producer and consumer

[ https://issues.apache.org/jira/browse/KAFKA-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-2598: --- Priority: Blocker (was: Major) > Add Test with authorizer for producer and consumer > ---

Java KafkaClient

Hi, We are just trying to use one of your Kafka Clients: KafkaConsumer (org.apache.kafka.clients.consumer) and was wondering why no Messages are returned by the poll method described in your examples... Then, I saw your implementation of the poll method and was a bit confused: /** * Fetc

[jira] [Commented] (KAFKA-2600) Make KStream interfaces compatible with Java 8 java.util.function

[ https://issues.apache.org/jira/browse/KAFKA-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941220#comment-14941220 ] Randall Hauch commented on KAFKA-2600: -- My original suggestion was to make these chan

[jira] [Updated] (KAFKA-2600) Make KStream interfaces compatible with Java 8 java.util.function

[ https://issues.apache.org/jira/browse/KAFKA-2600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Randall Hauch updated KAFKA-2600: - Description: As suggested by [~rhauch], if we make the interface method names as the same to java

[jira] [Commented] (KAFKA-2607) Review `Time` interface and its usage

[ https://issues.apache.org/jira/browse/KAFKA-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941066#comment-14941066 ] Ismael Juma commented on KAFKA-2607: We should probably do this after KAFKA-2606 to ma

[jira] [Created] (KAFKA-2607) Review `Time` interface and its usage

Ismael Juma created KAFKA-2607: -- Summary: Review `Time` interface and its usage Key: KAFKA-2607 URL: https://issues.apache.org/jira/browse/KAFKA-2607 Project: Kafka Issue Type: Improvement A

[jira] [Updated] (KAFKA-2606) Remove kafka.utils.Time in favour of o.a.kafka.common.utils.Time

[ https://issues.apache.org/jira/browse/KAFKA-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2606: --- Issue Type: Improvement (was: Bug) > Remove kafka.utils.Time in favour of o.a.kafka.common.utils.Time

[jira] [Created] (KAFKA-2606) Remove kafka.utils.Time in favour of o.a.kafka.common.utils.Time

Ismael Juma created KAFKA-2606: -- Summary: Remove kafka.utils.Time in favour of o.a.kafka.common.utils.Time Key: KAFKA-2606 URL: https://issues.apache.org/jira/browse/KAFKA-2606 Project: Kafka I

[jira] [Updated] (KAFKA-2605) Replace `catch: Throwable` clauses with `NonFatal` or `NonControl`

[ https://issues.apache.org/jira/browse/KAFKA-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2605: --- Issue Type: Improvement (was: Bug) > Replace `catch: Throwable` clauses with `NonFatal` or `NonContro

[jira] [Commented] (KAFKA-2605) Replace `catch: Throwable` clauses with `NonFatal` or `NonControl`

[ https://issues.apache.org/jira/browse/KAFKA-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941054#comment-14941054 ] Ismael Juma commented on KAFKA-2605: cc [~jjkoshy] > Replace `catch: Throwable` claus

[jira] [Updated] (KAFKA-2605) Replace `catch: Throwable` clauses with `NonFatal` or `NonControl`

[ https://issues.apache.org/jira/browse/KAFKA-2605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2605: --- Labels: newbie (was: ) > Replace `catch: Throwable` clauses with `NonFatal` or `NonControl` > ---

[jira] [Created] (KAFKA-2605) Replace `catch: Throwable` clauses with `NonFatal` or `NonControl`

Ismael Juma created KAFKA-2605: -- Summary: Replace `catch: Throwable` clauses with `NonFatal` or `NonControl` Key: KAFKA-2605 URL: https://issues.apache.org/jira/browse/KAFKA-2605 Project: Kafka

[jira] [Commented] (KAFKA-2566) Improve Jenkins set-up

[ https://issues.apache.org/jira/browse/KAFKA-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941044#comment-14941044 ] Flavio Junqueira commented on KAFKA-2566: - I'm a member of the hudson-jobadmin gro

[jira] [Updated] (KAFKA-2517) Performance Regression post SSL implementation

[ https://issues.apache.org/jira/browse/KAFKA-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2517: --- Status: Patch Available (was: Open) [~junrao], can you please review? > Performance Regression post

[jira] [Commented] (KAFKA-2517) Performance Regression post SSL implementation

[ https://issues.apache.org/jira/browse/KAFKA-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940993#comment-14940993 ] ASF GitHub Bot commented on KAFKA-2517: --- GitHub user ijuma opened a pull request:

[GitHub] kafka pull request: KAFKA-2517; Performance Regression post SSL im...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/273 KAFKA-2517; Performance Regression post SSL implementation (zero copy) You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijuma/kafka kafka-2517-ssl-

[jira] [Commented] (KAFKA-2517) Performance Regression post SSL implementation

[ https://issues.apache.org/jira/browse/KAFKA-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940991#comment-14940991 ] Ismael Juma commented on KAFKA-2517: Talked to Ben about this and he said it was OK fo

[jira] [Assigned] (KAFKA-2517) Performance Regression post SSL implementation

[ https://issues.apache.org/jira/browse/KAFKA-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reassigned KAFKA-2517: -- Assignee: Ismael Juma (was: Ben Stopford) > Performance Regression post SSL implementation > -

[jira] [Commented] (KAFKA-2604) Remove `completeAll` and improve timeout passed to `Selector.poll` from `NetworkClient.poll`

[ https://issues.apache.org/jira/browse/KAFKA-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940960#comment-14940960 ] Ismael Juma commented on KAFKA-2604: PR link: https://github.com/apache/kafka/pull/272

[jira] [Updated] (KAFKA-2604) Remove `completeAll` and improve timeout passed to `Selector.poll` from `NetworkClient.poll`

[ https://issues.apache.org/jira/browse/KAFKA-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2604: --- Reviewer: Jason Gustafson Status: Patch Available (was: In Progress) Jason, can you please revi

[GitHub] kafka pull request: KAFKA-2640; Remove `completeAll` and improve t...

GitHub user ijuma opened a pull request: https://github.com/apache/kafka/pull/272 KAFKA-2640; Remove `completeAll` and improve timeout passed to `Selector.poll` from `NetworkClient.poll` You can merge this pull request into a Git repository by running: $ git pull https://gith

[jira] [Work started] (KAFKA-2604) Remove `completeAll` and improve timeout passed to `Selector.poll` from `NetworkClient.poll`

[ https://issues.apache.org/jira/browse/KAFKA-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-2604 started by Ismael Juma. -- > Remove `completeAll` and improve timeout passed to `Selector.poll` from > `Network

[jira] [Created] (KAFKA-2604) Remove `completeAll` and improve timeout passed to `Selector.poll` from `NetworkClient.poll`

Ismael Juma created KAFKA-2604: -- Summary: Remove `completeAll` and improve timeout passed to `Selector.poll` from `NetworkClient.poll` Key: KAFKA-2604 URL: https://issues.apache.org/jira/browse/KAFKA-2604

[jira] [Updated] (KAFKA-1326) New consumer checklist

[ https://issues.apache.org/jira/browse/KAFKA-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-1326: --- Affects Version/s: (was: 0.10.0.0) 0.8.2.1 > New consumer checklist > -

[jira] [Reopened] (KAFKA-2587) Transient test failure: `SimpleAclAuthorizerTest`

[ https://issues.apache.org/jira/browse/KAFKA-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reopened KAFKA-2587: As Jason said, this is still happening. > Transient test failure: `SimpleAclAuthorizerTest` > -

[jira] [Updated] (KAFKA-2431) Test SSL/TLS impact on performance

[ https://issues.apache.org/jira/browse/KAFKA-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-2431: --- Reviewer: Jun Rao > Test SSL/TLS impact on performance > -- > >