Re: Help with Mirror Maker config to produce to all partitions in target cluster

Any exceptions in the mirror maker logs? may be you can enable mirror maker trace logs. maybe all messages are having same key? Can you recheck partition count on target cluster? On Thu, Nov 2, 2017 at 2:45 AM, Chris Neal wrote: > Apologies for bumping my own post, but really

Reg. Kafka transactional producer and consumer

Hi All, I am trying to make a hello world example for Transactional Producer and trying to consume. I am doing all this in plain java. I can produce but consumer is not consuming message. I searched over other places and I found some people have same problem. Right now, I am using single

Re: [ANNOUNCE] Apache Kafka 1.0.0 Released

Congratulations Kafka team on the release. Happy to see Kafka reach this milestone. It has been a pleasure using Kafka and also interacting with the Kafka team. -Jaikiran On 01/11/17 7:57 PM, Guozhang Wang wrote: The Apache Kafka community is pleased to announce the release for Apache Kafka

Re: [ANNOUNCE] Apache Kafka 1.0.0 Released

Good job, everyone! And thanks Guozhang for running the release! -James > On Nov 1, 2017, at 7:27 AM, Guozhang Wang wrote: > > The Apache Kafka community is pleased to announce the release for Apache > Kafka 1.0.0. > > This is a major release of the Kafka project, and is

Authentication using ssl in kafka

Hi, I am trying to configure apache kafka version 0.11.0.1 with ssl, but am faing the following error while trying to produce: Could you please suggest as to what can I do such that I can have client authentication taking place via ssl, my kafka configrations are given below:

Re: Building news feed of social app using kafka

Nope, that's the wrong design. It does not scale. You would end up in a wide and shallow thing. To few messages per partition to make sense. You want many thousands per partition per second to amortize the consumer to broker round-trip. On Nov 1, 2017 21:12, "Anshuman Ghosh"

Re: Help with Mirror Maker config to produce to all partitions in target cluster

Apologies for bumping my own post, but really hoping someone has experience with MirrorMaker who might be able to help with my question below. :) Thanks! Chris On Fri, Oct 27, 2017 at 10:59 AM, Chris Neal wrote: > Hi all, > > I've been struggling with this for awhile and need

Building news feed of social app using kafka

Hello! I am currently designing a social app (with the whole gamut of users following each other and personal feeds - consisting of posts by those you follow). To implement this "news feed" for each user, I was considering having a Kafka stream/topic per user. Given that our intention is to get

Re: Client Offset Deleted by Broker without cause

Right, they are: http://cupenya.com/files/tmp/hgd23121/consumer-logs.png http://cupenya.com/files/tmp/hgd23121/kafka-broker-logs.png (it's a bit more than in the pasted mail) On 11/01/2017 5:35 PM, Ted Yu wrote: bq. attached screenshots from the log viewer The screenshots didn't go through.

How do I instantiate a metrics reporter in Kafka Streams, with custom config?

Hi, we have a KafkaStreams app. We specify a custom metric reporter by doing: Properties config = new Properties(); config.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "kafka-broker1:9092"); config.put(StreamsConfig.METRIC_REPORTER_CLASSES_CONFIG,

Re: Client Offset Deleted by Broker without cause

bq. attached screenshots from the log viewer The screenshots didn't go through. Consider using 3rd party site. On Wed, Nov 1, 2017 at 9:18 AM, Elmar Weber wrote: > Hello, > > I had this morning the issue that a client offset got deleted from a > broker as it seems. > >

Client Offset Deleted by Broker without cause

Hello, I had this morning the issue that a client offset got deleted from a broker as it seems. (Kafka 0.11.0.1 with patch for KAFKA-6030 on top) It happened in a test environment and the pipeline stage got re-deployed multiple times, during one restart, when the consumer reconnected, it

The internals of commitSync failure

Is it possible that an exception returned by a consumer on commitSync could internally succeed for one or more partitions but fail for others and return or is it all or none of the partitions? Thanks, Kris

[ANNOUNCE] Apache Kafka 1.0.0 Released

The Apache Kafka community is pleased to announce the release for Apache Kafka 1.0.0. This is a major release of the Kafka project, and is no mere bump of the version number. The Apache Kafka Project Management Committee has packed a number of valuable enhancements into the release. Let me

Re: WordCount Example using GlobalKStore

Count will always use a StateStore, but if you want you can use an InMemory store if you don't want a persistent store. You can do this by using the overloaded `count(StateStoreSupplier)` method. You would use `Stores.create(name).inMemory()...` to create the inmemory store On Wed, 1 Nov 2017 at

WordCount Example using GlobalKStore

i have created 3 inputtopics with 10 partitions each and output Topic with 10 partitions I did wordcount example and stored it in GlobalKTable. i initally stored counted value in LocalStateStore and then it to GlobalStateStore. i have atteated the code here:

GlobalKStore

i have created 3 topics with 10 partitions each i have intended to store processed information in globalKtable now i have did with individual Ktable to Output topic then to GlobalKtable #// KStreamBuilder builder=new KStreamBuilder(); KStream inputStream =

Re: kafka user

Please follow the instructions listed here https://kafka.apache.org/contact You have to send a mail to 'users-subscr...@kafka.apache.org' to subscribe to user list. On Tue, Oct 31, 2017 at 1:06 PM, Karthigeyan wrote: > Hi , > > pls add to the user group. > >