[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14395955#comment-14395955 ] Jay Kreps commented on KAFKA-1729: -- [~jjkoshy] is this still waiting on an update from you? > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2.0 > > Attachments: KAFKA-1729.patch, KAFKA-1729.patch, > KAFKA-1729_2015-02-18_17:30:37.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14328493#comment-14328493 ] Joel Koshy commented on KAFKA-1729: --- Need to also update the protocol guide wiki with later versions of OCR/OFR > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2.0 > > Attachments: KAFKA-1729.patch, KAFKA-1729.patch, > KAFKA-1729_2015-02-18_17:30:37.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14327524#comment-14327524 ] Joel Koshy commented on KAFKA-1729: --- It has been pushed to trunk, but I will keep this open as I still have a draft for the doc update. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2.0 > > Attachments: KAFKA-1729.patch, KAFKA-1729.patch, > KAFKA-1729_2015-02-18_17:30:37.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326883#comment-14326883 ] Jun Rao commented on KAFKA-1729: Thanks for the patch. Pushed the change to 0.8.2. Could you push it to trunk and then close the ticket? > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2.0 > > Attachments: KAFKA-1729.patch, KAFKA-1729.patch, > KAFKA-1729_2015-02-18_17:30:37.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326862#comment-14326862 ] Joel Koshy commented on KAFKA-1729: --- Updated reviewboard https://reviews.apache.org/r/31169/diff/ against branch origin/0.8.2 > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2.0 > > Attachments: KAFKA-1729.patch, KAFKA-1729.patch, > KAFKA-1729_2015-02-18_17:30:37.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326764#comment-14326764 ] Joel Koshy commented on KAFKA-1729: --- Created reviewboard https://reviews.apache.org/r/31169/diff/ against branch origin/0.8.2 > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2.0 > > Attachments: KAFKA-1729.patch, KAFKA-1729.patch, > KAFKA-1782-doc-v1.patch, KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14309817#comment-14309817 ] Joel Koshy commented on KAFKA-1729: --- I will be making a few more edits after some feedback from [~jbrosenb...@gmail.com] > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1729.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303446#comment-14303446 ] Jun Rao commented on KAFKA-1729: [~jjkoshy], for 0.8.2, we can probably just update the doc to make it clear that using Kafka-based offset management at the SimpleConsumer level is not supported and remove the wiki page at https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka. Given that we are working on the new consumer, I am not sure if we need to support Kafka-based offset management at SimpleConsumer. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1729.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14302676#comment-14302676 ] Joel Koshy commented on KAFKA-1729: --- I committed the patch to trunk - but then realized an unfortunate mistake in the 0.8.2 patch above. The OffsetFetchRequest also hard-codes versionId to 0 so unfortunately we won't be able to fetch offsets from javaapi in 0.8.2 I'll fix it in trunk but I will mark this as closed. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1729.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14301864#comment-14301864 ] Joel Koshy commented on KAFKA-1729: --- Yes I will commit it today. Re: rolling back to ZK - yes that should work. A potential caveat is that we do filtered commits to ZK i.e., if an offset has not changed then don't commit. However, on the offset fetch in dual-commit mode we take the maximum between zk and kafka and will get committed on the next offset commit so I think it should be fine. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1729.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14301736#comment-14301736 ] Jun Rao commented on KAFKA-1729: Joel, Thanks for the doc. Looks good. Just a few minor comments. a higher setting (e.g., 100-200) is recommended.. => a higher setting (e.g., 100-200) is recommended for production. The brokers periodically compact the offsets topic since it only needs to maintain the most recent offset commit. => The brokers periodically compact the offsets topic since it only needs to maintain the most recent offset commit per partition. Also, when migrating from Zookeeper based storage to Kafka based one, how would a user roll back? Should they set offset.storage to zookeeper and keep dual.commit to true? Could you commit you changes today? > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1729.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298059#comment-14298059 ] Joel Koshy commented on KAFKA-1729: --- Thanks Jun - will do. When you get a chance, take a look at the attached doc patch as well which incorporates your comments. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1729.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14296205#comment-14296205 ] Jun Rao commented on KAFKA-1729: Joel, thanks for the patch. +1 and committed to 0.8.2. The patch doesn't apply to trunk though. So I will let you rebase and commit to trunk. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1729.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294647#comment-14294647 ] Joel Koshy commented on KAFKA-1729: --- (Uploaded above patch since we will most likely need another RC for 0.8.2) > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1729.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294645#comment-14294645 ] Joel Koshy commented on KAFKA-1729: --- Created reviewboard https://reviews.apache.org/r/30355/diff/ against branch origin/0.8.2 > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1729.patch, KAFKA-1782-doc-v1.patch, > KAFKA-1782-doc-v2.patch, KAFKA-1782-doc-v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290132#comment-14290132 ] Jun Rao commented on KAFKA-1729: [~jjkoshy], thanks for the update. Yes, I agree that we don't need to patch 0.8.2 for that. Going forward, we probably should avoid changing what's in SimpleConsumer since it will be replaced by the new consumer soon. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1782-doc-v1.patch, KAFKA-1782-doc-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290013#comment-14290013 ] Joel Koshy commented on KAFKA-1729: --- So perhaps it is best to leave the code as is since we anyway want to move completely over to the protocol definitions in the clients package. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1782-doc-v1.patch, KAFKA-1782-doc-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290011#comment-14290011 ] Joel Koshy commented on KAFKA-1729: --- Ideally yes because it makes Java access unwieldy but not a blocker because I think these are pre-existing even in 0.8.1 and there are work-arounds. Here is a brief summary: * https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/javaapi/OffsetCommitResponse.scala The OffsetCommitResponse returns a map of [TopicAndPartition, (scala)Short]. JavaConversions apparently does not convert the individual values in the map to a Java short. E.g., if you change the signature of errors() in OffsetCommitResponse to return Map[TopicAndPartition, java.lang.Short] it won't compile. The correct fix I think is to do something similar to javaapi.FetchResponse - where we provide an explicit lookup method errorCode(topicAndPartition) and a hasError boolean method. Anyway, the workaround is to just fetch the object from the map and cast it to short. So (for example): (java.lang.Short) errors.get(topicAndPartition) * A lot of the requests don't provide default for fields such as clientId, correlationId and even version. Well there are defaults but the defaults are inaccessible from Java. We should ideally provide alternate constructors. * It would be useful to add readFrom wrapper methods in the javaapi versions. Otherwise if you want to read from a blocking channel you would need to do something like this: new kafka.javaapi.ConsumerMetadataResponse(kafka.api.ConsumerMetadataResponse.readFrom(channel.receive().buffer())); > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1782-doc-v1.patch, KAFKA-1782-doc-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14289595#comment-14289595 ] Jun Rao commented on KAFKA-1729: Do you mean that you need to make code changes in the 0.8.2 branch or just the doc change? > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1782-doc-v1.patch, KAFKA-1782-doc-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288933#comment-14288933 ] Joel Koshy commented on KAFKA-1729: --- I was trying out the documentation I wrote on https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka but there are a couple of minor issues. I may need to make small changes to the javaapi versions of the request/responses. Will update tomorrow with a patch if necessary. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1782-doc-v1.patch, KAFKA-1782-doc-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287949#comment-14287949 ] Joel Koshy commented on KAFKA-1729: --- Should be able to update the doc patch today. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1782-doc-v1.patch, KAFKA-1782-doc-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14286651#comment-14286651 ] Jun Rao commented on KAFKA-1729: [~jjkoshy], are you on track to complete the doc for the 0.8.2 release? > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Fix For: 0.8.2 > > Attachments: KAFKA-1782-doc-v1.patch, KAFKA-1782-doc-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14209071#comment-14209071 ] Jun Rao commented on KAFKA-1729: Thanks for the patch. A few comments. 1. We need to make sure that before people start using the Kafka-based offset management in production, they set offsets.topic.num.partitions and offsets.topic.replication.factor properly for the offset topic since the defaults are not suitable for production usage. Could you add that in implementation.html? 2. It seems that issuing manual OffsetCommitRequest is only needed when using SimpleConsumer. We can probably make that clear in the wiki. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Attachments: KAFKA-1782-doc-v1.patch, KAFKA-1782-doc-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14199616#comment-14199616 ] Joel Koshy commented on KAFKA-1729: --- Incorporated comments. I would actually like to get rid of most of that wiki page I added (that is referenced from the doc) and recommend that people use the new OffsetClient utility that is being added in KAFKA-1013. I think it would be good to incorporate that into 0.8.2. I will review that patch again. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Attachments: KAFKA-1782-doc-v1.patch, KAFKA-1782-doc-v2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197243#comment-14197243 ] Joel Koshy commented on KAFKA-1729: --- Thanks for reviewing. Re: offsets in ZK Yes we can retain that, but we would like to remove that support altogether in a future release. I was hoping that this release would describe how to migrate off ZooKeeper and then in a subsequent 0.8.3 or 0.9 release we would remove that code altogether. We can of course keep it around for the next couple of releases but at some point we would like to remove it. Adding an example is a good idea. KAFKA-1013 actually contains an OffsetClient utility that helps but it won't be in 0.8.2 > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Attachments: KAFKA-1782-doc-v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-1729) add doc for Kafka-based offset management in 0.8.2
[ https://issues.apache.org/jira/browse/KAFKA-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197067#comment-14197067 ] Gwen Shapira commented on KAFKA-1729: - Since tracking consumer offsets in ZK is still an option in 0.8.2, I think we should keep this part of the doc around. Also, perhaps an example of how to commit with OffsetManager (including finding the OffsetManager)? Should be pretty simple, but I couldn't find any code examples. > add doc for Kafka-based offset management in 0.8.2 > -- > > Key: KAFKA-1729 > URL: https://issues.apache.org/jira/browse/KAFKA-1729 > Project: Kafka > Issue Type: Sub-task >Reporter: Jun Rao >Assignee: Joel Koshy > Attachments: KAFKA-1782-doc-v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
