Kafka consumer issue

@Team, I am trying to connect and consume from remote kafka, i am able to get number of partitions but when tried to consume from it not getting any data either start consuming from beginning or from latest. We have multiple topic in this kafka cluster, and there are consumer running for other

Build failed in Jenkins: kafka-trunk-jdk11 #188

See Changes: [wangguoz] KAFKA-7768: Use absolute paths for javadoc (#6100) -- [...truncated 2.25 MB...] org.apache.kafka.streams.test.OutputVerifierTest > shouldFailIfKey

Build failed in Jenkins: kafka-trunk-jdk8 #3291

See Changes: [github] KAFKA-7051: Improve the efficiency of ReplicaManager (#5206) [cmccabe] KAFKA-7051: Improve the efficiency of ReplicaManager (fixup) [wangguoz] KAFKA-7768: Use absolute paths for javadoc (#

Re: [DISCUSS] KIP-360: Improve handling of unknown producer

Thanks Jason. The proposed solution sounds good to me. Guozhang On Mon, Jan 7, 2019 at 3:52 PM Jason Gustafson wrote: > Hey Guozhang, > > Thanks for sharing the article. The INVALID_PRODUCER_ID_MAPPING error > occurs following expiration of the producerId. It's possible that another > producer

Re: [DISCUSS] KIP-360: Improve handling of unknown producer

Hey Guozhang, Thanks for sharing the article. The INVALID_PRODUCER_ID_MAPPING error occurs following expiration of the producerId. It's possible that another producerId has been installed in its place following expiration (if another producer instance has become active), or the mapping is empty. W

Re: [VOTE] KIP-382 MirrorMaker 2.0

Thanks Jun, I've updated the KIP as requested. Brief notes below: 100. added "...out-of-the-box (without custom handlers)..." 101. done. Good idea to include a MessageFormatter. 102. done. > 103. [...] why is Heartbeat a separate connector? Heartbeats themselves are replicated via MirrorSource

Build failed in Jenkins: kafka-trunk-jdk11 #187

See Changes: [github] KAFKA-7051: Improve the efficiency of ReplicaManager (#5206) [cmccabe] KAFKA-7051: Improve the efficiency of ReplicaManager (fixup) -- [...truncated

Build failed in Jenkins: kafka-trunk-jdk8 #3290

See Changes: [wangguoz] KAFKA-7768: Add version to java html urls (#6094) -- [...truncated 4.50 MB...] org.apache.kafka.streams.test.OutputVerifierTest > shouldFailIfValu

Re: KIP-213 - Scalable/Usable Foreign-Key KTable joins - Rebooted.

On 02.01.2019 23:44, John Roesler wrote: > However, you seem to have a strong intuition that the scatter/gather > approach is better. > Is this informed by your actual applications at work? Perhaps you can > provide an example > data set and sequence of operations so we can all do the math and ag

Re: Contributor Add Request

Hi, Radwa. Thanks for your interest. Added you to the contributors list. Jun On Mon, Jan 7, 2019 at 1:49 PM Radwa Osama wrote: > Hello All, > > Would you please add me to contributors list? > > My jira username is: RO86 > > Thanks > Radwa >

[jira] [Created] (KAFKA-7793) Improve the Trogdor command-line

Colin P. McCabe created KAFKA-7793: -- Summary: Improve the Trogdor command-line Key: KAFKA-7793 URL: https://issues.apache.org/jira/browse/KAFKA-7793 Project: Kafka Issue Type: Improvement

Contributor Add Request

Hello All, Would you please add me to contributors list? My jira username is: RO86 Thanks Radwa

Re: [DISCUSS] KIP-391: Allow Producing with Offsets for Cluster Replication

Hi, I delayed starting the voting thread due to the festive period. I would like to start it this week. Has anyone any more feedback ? -- Edoardo Comar IBM Event Streams Edoardo Comar wrote on 13/12/2018 17:50:30: > From: Edoardo Comar > To:

[jira] [Resolved] (KAFKA-4218) Enable access to key in ValueTransformer

[ https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4218. -- Resolution: Fixed Fix Version/s: (was: 1.1.0) 2.1.0 This has been

[jira] [Resolved] (KAFKA-4468) Correctly calculate the window end timestamp after read from state stores

[ https://issues.apache.org/jira/browse/KAFKA-4468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-4468. -- Resolution: Fixed Assignee: Richard Yu > Correctly calculate the window end timestamp aft

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

Hey Stanislav, I think the time taken to rebalance is not linearly correlated with number of consumers with our application. As for our current and future use cases, the main concern for Pinterest is still on the broker memory not CPU, because crashing server by one application could have cascad

Build failed in Jenkins: kafka-2.1-jdk8 #97

See Changes: [wangguoz] KAFKA-7768: Add version to java html urls (#6094) -- [...truncated 462.06 KB...] kafka.security.auth.SimpleAclAuthorizerTest > testAddAclsOnWildcardRe

[jira] [Created] (KAFKA-7792) Trogdor should have an uptime function

Colin P. McCabe created KAFKA-7792: -- Summary: Trogdor should have an uptime function Key: KAFKA-7792 URL: https://issues.apache.org/jira/browse/KAFKA-7792 Project: Kafka Issue Type: Improvem

[jira] [Resolved] (KAFKA-7755) Kubernetes - Kafka clients are resolving DNS entries only one time

[ https://issues.apache.org/jira/browse/KAFKA-7755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajini Sivaram resolved KAFKA-7755. --- Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 > Kubernetes - Kafka c

Re: [DISCUSS] KIP-411: Add option to make Kafka Connect task client ID values unique

Thanks all. I've submitted a new PR with a possible implementation: https://github.com/apache/kafka/pull/6097. Note I did not include the group ID as part of the default client ID, mainly to avoid the connector name appearing twice by default. As noted in the original Jira ( https://issues.apache.o

Jenkins build is back to normal : kafka-trunk-jdk8 #3289

See

Re: [DISCUSS] KIP-411: Add option to make Kafka Connect task client ID values unique

I'd also prefer to avoid the new configuration property if possible. Seems like a lighter touch without it. Ryanne On Sun, Jan 6, 2019 at 7:25 PM Paul Davidson wrote: > Hi Konstantine, > > Thanks for your feedback! I think my reply to Ewen covers most of your > points, and I mostly agree. If

Re: [DISCUSS] KIP-382: MirrorMaker 2.0

Hi Ewen, thanks for the questions. > On the ACL management, can you explain how things are supposed to work... There are two types of topics in play here: regular topics and remote topics. MM2 replicates regular source topics -> remote topics. MM2 doesn't create or modify regular topics, but it

Re: [VOTE] KIP-228 Negative record timestamp support

I think it is true: currently timestamp field can only be non-negative or -1, and any other values will be rejected; this proposal allows negative values but did not change the semantics of current values (non-negative and -1). So for current users, if they still only use non-negative or -1 values

[jira] [Resolved] (KAFKA-7768) Java doc link 404

[ https://issues.apache.org/jira/browse/KAFKA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-7768. -- Resolution: Fixed Fix Version/s: 2.1.1 2.2.0 > Java doc link 404 > -

[jira] [Created] (KAFKA-7791) Not log retriable exceptions as errors

Yaroslav Klymko created KAFKA-7791: -- Summary: Not log retriable exceptions as errors Key: KAFKA-7791 URL: https://issues.apache.org/jira/browse/KAFKA-7791 Project: Kafka Issue Type: Bug

RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer

Hi guys, I am not sure what to do next in this KIP process. Could anyone please help/advise me on what to do next? Thanks, CH -Original Message- From: ChienHsing Wu Sent: Wednesday, January 02, 2019 12:55 PM To: dev@kafka.apache.org Subject: RE: [EXTERNAL] - Re: [DISCUSS] KIP-387: Fa

[jira] [Resolved] (KAFKA-7695) Cannot override StreamsPartitionAssignor in configuration

[ https://issues.apache.org/jira/browse/KAFKA-7695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Buykin resolved KAFKA-7695. -- Resolution: Not A Bug > Cannot override StreamsPartitionAssignor in configuration > --

[VOTE] - KIP-213 (new vote) - Simplified and revised.

Hi All I would like to call a new vote on KIP-213. The design has changed substantially. Perhaps more importantly, the KIP and associated documentation has been greatly simplified. I know this KIP has been on the mailing list for a long time, but the help from John Roesler and Guozhang Wang have h

[jira] [Created] (KAFKA-7790) Trogdor - Does not time out tasks in time

Stanislav Kozlovski created KAFKA-7790: -- Summary: Trogdor - Does not time out tasks in time Key: KAFKA-7790 URL: https://issues.apache.org/jira/browse/KAFKA-7790 Project: Kafka Issue Typ

[jira] [Created] (KAFKA-7789) SSL-related unit tests hang when run on Fedora 29

Tom Bentley created KAFKA-7789: -- Summary: SSL-related unit tests hang when run on Fedora 29 Key: KAFKA-7789 URL: https://issues.apache.org/jira/browse/KAFKA-7789 Project: Kafka Issue Type: Bug

Re: [DISCUSS] KIP-395: Encypt-then-MAC Delegation token metadata

Manikumar, Satish. Thanks for the review! As I understand, you are not in favor of this KIP, and I do agree that having a pluggable mechanism for sensitive data / metadata is preferable/more future-proof. On Wed, Dec 12, 2018 at 8:12 AM Satish Duggana wrote: > Agree with Manikumar on having plug

Build failed in Jenkins: kafka-trunk-jdk8 #3288

See Changes: [wangguoz] MINOR: Remove sleep calls and ignore annotation from streams upgrade -- [...truncated 4.50 MB...] org.apache.kafka.streams.test.OutputVerifierTest

Re: [Discuss] KIP-389: Enforce group.max.size to cap member metadata growth

Hey there, Per Gwen's comments, I slightly reworked the motivation section. Let me know if it's any better now I completely agree that it would be best if we were to add a recommended number to a typical consumer group size. There is a problem that timing the CPU usage and rebalance times of cons