[jira] [Updated] (KAFKA-5282) Transactions integration test: Use factory methods to keep track of open producers and consumers and close them all on tearDown

2017-05-25 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vahid Hashemian updated KAFKA-5282: --- Status: Patch Available (was: In Progress) > Transactions integration test: Use factory metho

[jira] [Updated] (KAFKA-5331) MirrorMaker can't guarantee data is consistent between clusters

2017-05-25 Thread xuzq (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xuzq updated KAFKA-5331: Attachment: KAFKA-5331.patch > MirrorMaker can't guarantee data is consistent between clusters > ---

[jira] [Updated] (KAFKA-5281) System tests for KIP-98 / transactions

2017-05-25 Thread Apurva Mehta (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apurva Mehta updated KAFKA-5281: Status: Patch Available (was: In Progress) > System tests for KIP-98 / transactions > -

[jira] [Assigned] (KAFKA-5251) Producer should drop queued sends when transaction is aborted

2017-05-25 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson reassigned KAFKA-5251: -- Assignee: Jason Gustafson > Producer should drop queued sends when transaction is abort

[jira] [Created] (KAFKA-5331) MirrorMaker can't guarantee data is consistent between clusters

2017-05-25 Thread xuzq (JIRA)
xuzq created KAFKA-5331: --- Summary: MirrorMaker can't guarantee data is consistent between clusters Key: KAFKA-5331 URL: https://issues.apache.org/jira/browse/KAFKA-5331 Project: Kafka Issue Type: Impr

[jira] [Commented] (KAFKA-5281) System tests for KIP-98 / transactions

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025865#comment-16025865 ] ASF GitHub Bot commented on KAFKA-5281: --- GitHub user apurvam opened a pull request:

[GitHub] kafka pull request #3149: KAFKA-5281: System tests for transactions

2017-05-25 Thread apurvam
GitHub user apurvam opened a pull request: https://github.com/apache/kafka/pull/3149 KAFKA-5281: System tests for transactions You can merge this pull request into a Git repository by running: $ git pull https://github.com/apurvam/kafka KAFKA-5281-transactions-system-tests A

[jira] [Created] (KAFKA-5330) Use per-task converters in Connect

2017-05-25 Thread Ewen Cheslack-Postava (JIRA)
Ewen Cheslack-Postava created KAFKA-5330: Summary: Use per-task converters in Connect Key: KAFKA-5330 URL: https://issues.apache.org/jira/browse/KAFKA-5330 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-5329) Replica list in the metadata cache on the broker may have different order from zookeeper

2017-05-25 Thread Jiangjie Qin (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiangjie Qin updated KAFKA-5329: Summary: Replica list in the metadata cache on the broker may have different order from zookeeper (

[jira] [Created] (KAFKA-5329) Metadata cache on the broker may have replica list may have different order from zookeeper

2017-05-25 Thread Jiangjie Qin (JIRA)
Jiangjie Qin created KAFKA-5329: --- Summary: Metadata cache on the broker may have replica list may have different order from zookeeper Key: KAFKA-5329 URL: https://issues.apache.org/jira/browse/KAFKA-5329

[jira] [Created] (KAFKA-5328) consider switching json parser from scala to jackson

2017-05-25 Thread Onur Karaman (JIRA)
Onur Karaman created KAFKA-5328: --- Summary: consider switching json parser from scala to jackson Key: KAFKA-5328 URL: https://issues.apache.org/jira/browse/KAFKA-5328 Project: Kafka Issue Type:

Build failed in Jenkins: kafka-trunk-jdk8 #1596

2017-05-25 Thread Apache Jenkins Server
See -- [...truncated 2.94 MB...] kafka.security.auth.SimpleAclAuthorizerTest > testDistributedConcurrentModificationOfResourceAcls PASSED kafka.security.auth.SimpleAclAuthorizerTest >

[jira] [Commented] (KAFKA-4585) Offset fetch and commit requests use the same permissions

2017-05-25 Thread Ewen Cheslack-Postava (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025746#comment-16025746 ] Ewen Cheslack-Postava commented on KAFKA-4585: -- [~vahid] Honestly not sure --

[jira] [Commented] (KAFKA-5327) Console Consumer should only poll for up to max messages

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025743#comment-16025743 ] ASF GitHub Bot commented on KAFKA-5327: --- GitHub user amethystic opened a pull reques

[GitHub] kafka pull request #3148: KAFKA-5327: ConsoleConsumer explicitly set `max.po...

2017-05-25 Thread amethystic
GitHub user amethystic opened a pull request: https://github.com/apache/kafka/pull/3148 KAFKA-5327: ConsoleConsumer explicitly set `max.poll.records`... KAFKA-5327: ConsoleConsumer explicitly set `max.poll.records` if `--max-messages` is set. You can merge this pull request into a

[jira] [Assigned] (KAFKA-5327) Console Consumer should only poll for up to max messages

2017-05-25 Thread huxi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huxi reassigned KAFKA-5327: --- Assignee: huxi > Console Consumer should only poll for up to max messages > -

[jira] [Commented] (KAFKA-5226) NullPointerException (NPE) in SourceNodeRecordDeserializer.deserialize

2017-05-25 Thread Ian Springer (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025688#comment-16025688 ] Ian Springer commented on KAFKA-5226: - Thanks for the update, [~bbejeck]. That's great

[jira] [Updated] (KAFKA-5320) Update produce/fetch throttle time metrics for any request throttle

2017-05-25 Thread Jun Rao (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao updated KAFKA-5320: --- Fix Version/s: (was: 0.11.1.0) > Update produce/fetch throttle time metrics for any request throttle > ---

RE: GlobalKTable limitations

2017-05-25 Thread Thomas Becker
Hey Eno, Thanks for the response. We have considered, but not yet tried that. One of the nice things about the GlobalKTable is that it fully "bootstraps" before the rest of the topology is started. But if part of the topology is itself generating the topic that backs the global table, it seems l

[jira] [Commented] (KAFKA-4585) Offset fetch and commit requests use the same permissions

2017-05-25 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025569#comment-16025569 ] Vahid Hashemian commented on KAFKA-4585: [~ewencp] I'll work on a KIP for changing

[jira] [Commented] (KAFKA-5147) KafkaProducer's TransactionManager needs a review on synchronization

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025552#comment-16025552 ] ASF GitHub Bot commented on KAFKA-5147: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #3132: KAFKA-5147: Add missing synchronization to Transac...

2017-05-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3132 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-5147) KafkaProducer's TransactionManager needs a review on synchronization

2017-05-25 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-5147: --- Resolution: Fixed Fix Version/s: 0.11.1.0 Status: Resolved (was: Patch Avail

[jira] [Work started] (KAFKA-4585) Offset fetch and commit requests use the same permissions

2017-05-25 Thread Vahid Hashemian (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on KAFKA-4585 started by Vahid Hashemian. -- > Offset fetch and commit requests use the same permissions > ---

[GitHub] kafka pull request #3147: MINOR: Remove unused method parameter in `SimpleAc...

2017-05-25 Thread vahidhashemian
GitHub user vahidhashemian opened a pull request: https://github.com/apache/kafka/pull/3147 MINOR: Remove unused method parameter in `SimpleAclAuthorizer` You can merge this pull request into a Git repository by running: $ git pull https://github.com/vahidhashemian/kafka mino

[jira] [Updated] (KAFKA-5226) NullPointerException (NPE) in SourceNodeRecordDeserializer.deserialize

2017-05-25 Thread Bill Bejeck (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck updated KAFKA-5226: --- Attachment: streamsAppTwo.log streamsAppThree.log streamsAppOne.log >

[jira] [Commented] (KAFKA-5226) NullPointerException (NPE) in SourceNodeRecordDeserializer.deserialize

2017-05-25 Thread Bill Bejeck (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025404#comment-16025404 ] Bill Bejeck commented on KAFKA-5226: I've been able to reproduce the problem. The iss

[jira] [Commented] (KAFKA-5265) Move ACLs, Config, NodeVersions classes into org.apache.kafka.common

2017-05-25 Thread Colin P. McCabe (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025356#comment-16025356 ] Colin P. McCabe commented on KAFKA-5265: CreateTopicResults -> CreateTopicsResults

Jenkins build is back to normal : kafka-trunk-jdk8 #1593

2017-05-25 Thread Apache Jenkins Server
See

Re: Kafka Read Data from All Partition Using Key or Timestamp

2017-05-25 Thread SenthilKumar K
Thanks a lot Hans.. By using KafkaConsumer API https://gist.github.com/senthilec566/16e8e28b32834666fea132afc3a4e2f9 i can query the data using timestamp .. It worked ! Now another question to achieve Parallelism on reading data .. Example : topic : test partitions : 4 Kafk

[jira] [Commented] (KAFKA-3925) Default log.dir=/tmp/kafka-logs is unsafe

2017-05-25 Thread Scott Glajch (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-3925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025259#comment-16025259 ] Scott Glajch commented on KAFKA-3925: - This really bit us as well. I don't know why y

[jira] [Resolved] (KAFKA-5320) Update produce/fetch throttle time metrics for any request throttle

2017-05-25 Thread Rajini Sivaram (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-5320. --- Resolution: Fixed Fix Version/s: 0.11.1.0 Issue resolved by pull request 3137 [https://

[jira] [Commented] (KAFKA-5320) Update produce/fetch throttle time metrics for any request throttle

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025256#comment-16025256 ] ASF GitHub Bot commented on KAFKA-5320: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #3137: KAFKA-5320: Include all request throttling in clie...

2017-05-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3137 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] kafka pull request #3146: MINOR: Cleanup in tests to avoid threads being lef...

2017-05-25 Thread rajinisivaram
GitHub user rajinisivaram opened a pull request: https://github.com/apache/kafka/pull/3146 MINOR: Cleanup in tests to avoid threads being left behind You can merge this pull request into a Git repository by running: $ git pull https://github.com/rajinisivaram/kafka MINOR-test-

Re: [DISCUSS]: KIP-161: streams record processing exception handlers

2017-05-25 Thread Avi Flax
> On May 25, 2017, at 14:27, Matthias J. Sax wrote: > > Thanks for the KIP Eno! Yes, thank you Eno! It’s very welcome and heartening to see work on this front. I have a few follow-ups to Matthias’ comments: > Why we introduce `ExceptionType` and not just hand in the actual exception? I was w

[GitHub] kafka pull request #3145: MINOR: Add test case for duplicate check after log...

2017-05-25 Thread hachikuji
GitHub user hachikuji opened a pull request: https://github.com/apache/kafka/pull/3145 MINOR: Add test case for duplicate check after log cleaning You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka minor-improve-bas

[jira] [Created] (KAFKA-5327) Console Consumer should only poll for up to max messages

2017-05-25 Thread Dustin Cote (JIRA)
Dustin Cote created KAFKA-5327: -- Summary: Console Consumer should only poll for up to max messages Key: KAFKA-5327 URL: https://issues.apache.org/jira/browse/KAFKA-5327 Project: Kafka Issue Type

[jira] [Updated] (KAFKA-5323) AdminUtils.createTopic should check topic existence upfront

2017-05-25 Thread Onur Karaman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Onur Karaman updated KAFKA-5323: Status: Patch Available (was: Open) > AdminUtils.createTopic should check topic existence upfront >

Re: [DISCUSS]: KIP-161: streams record processing exception handlers

2017-05-25 Thread Matthias J. Sax
Thanks for the KIP Eno! Couple of comments: I think we don't need `RecordContext` in `RecordExceptionHandler#handle` because the `ConsumerRecord` provides all this information anyway. Why we introduce `ExceptionType` and not just hand in the actual exception? As return type of `handle()` is voi

[GitHub] kafka pull request #3144: KAFKA-5323: AdminUtils.createTopic should check to...

2017-05-25 Thread onurkaraman
GitHub user onurkaraman opened a pull request: https://github.com/apache/kafka/pull/3144 KAFKA-5323: AdminUtils.createTopic should check topic existence upfront When a topic exists, AdminUtils.createTopic unnecessarily does N+2 zookeeper reads where N is the number of brokers. Here

[jira] [Commented] (KAFKA-5323) AdminUtils.createTopic should check topic existence upfront

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025147#comment-16025147 ] ASF GitHub Bot commented on KAFKA-5323: --- GitHub user onurkaraman opened a pull reque

[jira] [Commented] (KAFKA-5279) TransactionCoordinator must expire transactionalIds

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025128#comment-16025128 ] ASF GitHub Bot commented on KAFKA-5279: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #3101: KAFKA-5279: TransactionCoordinator must expire tra...

2017-05-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3101 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Updated] (KAFKA-5279) TransactionCoordinator must expire transactionalIds

2017-05-25 Thread Guozhang Wang (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-5279: - Resolution: Fixed Fix Version/s: 0.11.0.0 Status: Resolved (was: Patch Available

[jira] [Updated] (KAFKA-5017) Consider making baseOffset the first offset in message format v2

2017-05-25 Thread Jason Gustafson (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-5017: --- Resolution: Won't Fix Status: Resolved (was: Patch Available) I had thought that this

[jira] [Commented] (KAFKA-5017) Consider making baseOffset the first offset in message format v2

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025074#comment-16025074 ] ASF GitHub Bot commented on KAFKA-5017: --- Github user hachikuji closed the pull reque

[GitHub] kafka pull request #3138: KAFKA-5017: Record batch first offset remains accu...

2017-05-25 Thread hachikuji
Github user hachikuji closed the pull request at: https://github.com/apache/kafka/pull/3138 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[jira] [Commented] (KAFKA-4362) Consumer can fail after reassignment of the offsets topic partition

2017-05-25 Thread Jeff Widman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024997#comment-16024997 ] Jeff Widman commented on KAFKA-4362: Updated the applicable version as we just encount

[jira] [Updated] (KAFKA-4362) Consumer can fail after reassignment of the offsets topic partition

2017-05-25 Thread Jeff Widman (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Widman updated KAFKA-4362: --- Affects Version/s: 0.10.0.1 > Consumer can fail after reassignment of the offsets topic partition > --

Re: GlobalKTable limitations

2017-05-25 Thread Eno Thereska
Hi Thomas, Have you considered doing the transformations on the topic, then outputting to another topic and then constructing the GlobalKTable from the latter? The GlobalKTable has the limitations you mention since it was primarily designed for joins only. We should consider allowing a less re

[jira] [Commented] (KAFKA-5154) Kafka Streams throws NPE during rebalance

2017-05-25 Thread Damian Guy (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024889#comment-16024889 ] Damian Guy commented on KAFKA-5154: --- [~Lukas Gemela] if it happens again can you please

Re: Kafka Read Data from All Partition Using Key or Timestamp

2017-05-25 Thread Hans Jespersen
The timeindex was added in 0.10 so I think you need to use the new Consumer API to access this functionality. Specifically you should call offsetsForTimes() https://kafka.apache.org/0102/javadoc/org/apache/kafka/clients/consumer/Consumer.html#offsetsForTimes(java.util.Map) -hans > On May 25, 20

[jira] [Comment Edited] (KAFKA-5326) Log offset index resize cause empty index file

2017-05-25 Thread xuzq (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024835#comment-16024835 ] xuzq edited comment on KAFKA-5326 at 5/25/17 3:04 PM: -- When the kafka

[jira] [Commented] (KAFKA-5326) Log offset index resize cause empty index file

2017-05-25 Thread xuzq (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024835#comment-16024835 ] xuzq commented on KAFKA-5326: - When the kafka server is running, someone removed the index fil

[jira] [Commented] (KAFKA-5326) Log offset index resize cause empty index file

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024759#comment-16024759 ] ASF GitHub Bot commented on KAFKA-5326: --- GitHub user ZanderXu reopened a pull reques

[GitHub] kafka pull request #3143: KAFKA-5326

2017-05-25 Thread ZanderXu
GitHub user ZanderXu reopened a pull request: https://github.com/apache/kafka/pull/3143 KAFKA-5326 This modify to resolved the [KAFKA-5326](https://issues.apache.org/jira/browse/KAFKA-5326). You can merge this pull request into a Git repository by running: $ git pull https://g

[jira] [Commented] (KAFKA-5326) Log offset index resize cause empty index file

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024755#comment-16024755 ] ASF GitHub Bot commented on KAFKA-5326: --- Github user ZanderXu closed the pull reques

[jira] [Commented] (KAFKA-5326) Log offset index resize cause empty index file

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024753#comment-16024753 ] ASF GitHub Bot commented on KAFKA-5326: --- GitHub user ZanderXu reopened a pull reques

[GitHub] kafka pull request #3143: KAFKA-5326

2017-05-25 Thread ZanderXu
Github user ZanderXu closed the pull request at: https://github.com/apache/kafka/pull/3143 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[jira] [Commented] (KAFKA-5326) Log offset index resize cause empty index file

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024752#comment-16024752 ] ASF GitHub Bot commented on KAFKA-5326: --- Github user ZanderXu closed the pull reques

[GitHub] kafka pull request #3143: KAFKA-5326

2017-05-25 Thread ZanderXu
GitHub user ZanderXu reopened a pull request: https://github.com/apache/kafka/pull/3143 KAFKA-5326 This modify to resolved the [KAFKA-5326](https://issues.apache.org/jira/browse/KAFKA-5326). You can merge this pull request into a Git repository by running: $ git pull https://g

[GitHub] kafka pull request #3143: KAFKA-5326

2017-05-25 Thread ZanderXu
Github user ZanderXu closed the pull request at: https://github.com/apache/kafka/pull/3143 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

GlobalKTable limitations

2017-05-25 Thread Thomas Becker
We need to do a series of joins against a KTable that we can't co- partition with the stream, so we're looking at GlobalKTable. But the topic backing the table is not ideally keyed for the sort of lookups this particular processor needs to do. Unfortunately, GlobalKTable is very limited in that yo

Re: Kafka Read Data from All Partition Using Key or Timestamp

2017-05-25 Thread SenthilKumar K
I did an experiment on searching messages using timestamps .. Step 1: Used Producer with Create Time ( CT ) Step 2 : Verify whether it reflects in Kafka or not .index .log .timeindex These three files in disk

[jira] [Commented] (KAFKA-5326) Log offset index resize cause empty index file

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024728#comment-16024728 ] ASF GitHub Bot commented on KAFKA-5326: --- GitHub user ZanderXu opened a pull request:

[GitHub] kafka pull request #3143: to resolved the KAFKA-5326

2017-05-25 Thread ZanderXu
GitHub user ZanderXu opened a pull request: https://github.com/apache/kafka/pull/3143 to resolved the KAFKA-5326 This modify to resolved the [KAFKA-5326](https://issues.apache.org/jira/browse/KAFKA-5326). You can merge this pull request into a Git repository by running: $ git

[jira] [Created] (KAFKA-5326) Log offset index resize cause empty index file

2017-05-25 Thread Ma Tianchi (JIRA)
Ma Tianchi created KAFKA-5326: - Summary: Log offset index resize cause empty index file Key: KAFKA-5326 URL: https://issues.apache.org/jira/browse/KAFKA-5326 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-5319) Add a tool to make cluster replica and leader balance

2017-05-25 Thread Ma Tianchi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ma Tianchi updated KAFKA-5319: -- Attachment: AlgorithmDescription.png > Add a tool to make cluster replica and leader balance > -

[jira] [Updated] (KAFKA-5319) Add a tool to make cluster replica and leader balance

2017-05-25 Thread Ma Tianchi (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ma Tianchi updated KAFKA-5319: -- Attachment: (was: ArithmeticDescription.png) > Add a tool to make cluster replica and leader balance

[jira] [Commented] (KAFKA-5263) kakfa-clients consume 100% CPU with manual partition assignment when network connection is lost

2017-05-25 Thread Konstantin Smirnov (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024593#comment-16024593 ] Konstantin Smirnov commented on KAFKA-5263: --- Thanks a lot for your fix! > kakfa

[GitHub] kafka pull request #3124: KAFKA-5263: Avoid tight polling loop in consumer w...

2017-05-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3124 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[jira] [Resolved] (KAFKA-5263) kakfa-clients consume 100% CPU with manual partition assignment when network connection is lost

2017-05-25 Thread Rajini Sivaram (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-5263. --- Resolution: Fixed Fix Version/s: 0.11.1.0 0.11.0.0 Issue resolved by

[jira] [Commented] (KAFKA-5263) kakfa-clients consume 100% CPU with manual partition assignment when network connection is lost

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024492#comment-16024492 ] ASF GitHub Bot commented on KAFKA-5263: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #3139: MINOR: fix flakiness in testDeleteAcls

2017-05-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3139 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: Kafka Read Data from All Partition Using Key or Timestamp

2017-05-25 Thread SenthilKumar K
Thanks a lot Mayuresh. I will look into SearchMessageByTimestamp feature in Kafka .. Cheers, Senthil On Thu, May 25, 2017 at 1:12 PM, Mayuresh Gharat wrote: > Hi Senthil, > > Kafka does allow search message by timestamp after KIP-33 : > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 3

[DISCUSS]: KIP-161: streams record processing exception handlers

2017-05-25 Thread Eno Thereska
Hi there, I’ve added a KIP on improving exception handling in streams: KIP-161: streams record processing exception handlers. https://cwiki.apache.org/confluence/display/KAFKA/KIP-161%3A+streams+record+processing+exception+handlers

Jenkins build is back to normal : kafka-trunk-jdk8 #1590

2017-05-25 Thread Apache Jenkins Server
See

[jira] [Commented] (KAFKA-5173) SASL tests failing with Could not find a 'KafkaServer' or 'sasl_plaintext.KafkaServer' entry in the JAAS configuration

2017-05-25 Thread liumei (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024439#comment-16024439 ] liumei commented on KAFKA-5173: --- When i start broker with sasl config, it failed. Is it the

[jira] [Updated] (KAFKA-4935) Consider disabling record level CRC checks for message format V2

2017-05-25 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4935: --- Fix Version/s: 0.11.0.0 > Consider disabling record level CRC checks for message format V2 > -

[jira] [Commented] (KAFKA-5211) KafkaConsumer should not skip a corrupted record after throwing an exception.

2017-05-25 Thread Eno Thereska (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024372#comment-16024372 ] Eno Thereska commented on KAFKA-5211: - Thanks [~becket_qin], [~ijuma], [~hachikuji]. M

Re: Kafka Read Data from All Partition Using Key or Timestamp

2017-05-25 Thread Mayuresh Gharat
Hi Senthil, Kafka does allow search message by timestamp after KIP-33 : https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index#KIP-33-Addatimebasedlogindex-Searchmessagebytimestamp The new consumer does provide you a way to get offsets by timestamp. You can use thes

[jira] [Created] (KAFKA-5325) Connection Lose during Kafka Kerberos Renewal process

2017-05-25 Thread MuthuKumar (JIRA)
MuthuKumar created KAFKA-5325: - Summary: Connection Lose during Kafka Kerberos Renewal process Key: KAFKA-5325 URL: https://issues.apache.org/jira/browse/KAFKA-5325 Project: Kafka Issue Type: Bug

[jira] [Updated] (KAFKA-4935) Consider disabling record level CRC checks for message format V2

2017-05-25 Thread Ismael Juma (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-4935: --- Resolution: Fixed Fix Version/s: (was: 0.11.0.0) 0.11.1.0 Stat

[jira] [Commented] (KAFKA-4935) Consider disabling record level CRC checks for message format V2

2017-05-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-4935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024335#comment-16024335 ] ASF GitHub Bot commented on KAFKA-4935: --- Github user asfgit closed the pull request

[GitHub] kafka pull request #3123: KAFKA-4935: Deprecate client checksum API and comp...

2017-05-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/3123 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

Re: Kafka Read Data from All Partition Using Key or Timestamp

2017-05-25 Thread SenthilKumar K
Hello Dev Team, Pls let me know if any option to read data from Kafka (all partition ) using timestamp . Also can we set custom offset value to messages ? Cheers, Senthil On Wed, May 24, 2017 at 7:33 PM, SenthilKumar K wrote: > Hi All , We have been using Kafka for our Use Case which helps in