Purging will never prevent that it does not get replicated for sure. There will
be always a case (error to purge etc) and then it is still replicated. You may
reduce the probability but it will never be impossible.
Your application should be able to handle duplicated messages.
> On 25. May 201
Hi Shantanu
I am not sure the scenario you are describing is the best case. I would
more consider the problem in term of producers and consumers of the data.
Usually is a good practice to put your producer local to your kafka
cluster, so in your case, I would suggest you have producers in the main
Hi Vincent,
Our producers are consumers are indeed local to Kafka cluster. When we
switch DC everything switches. So when we are on backup producers and
consumers on backup DC are active, everything on primary DC is stopped.
Whatever data gets accumulated on backup DC needs to be reflected in
pri
What is the end results done by your consumers ?
>From what I understand, having the need for no duplicates means that these
duplicates can show up somewhere ?
According your needs, you can also have consumers in the two DC consuming
from both. Then you don't have duplicate because a message is ei
subscribe mail list
To subscribe, please follow instructions here:
https://kafka.apache.org/contact
On 5/24/18 8:16 PM, wrote:
> subscribe mail list
>
signature.asc
Description: OpenPGP digital signature
Could anyone please help?
Am 21.05.2018 um 10:56 schrieb chw:
> Hi everybody,
>
> the communication between the client and the broker is unclear to me.
> The documentation states:
>
>> The client initiates a socket connection and then writes a sequence of
>> request messages and reads back the co
Hi Vincent,
We have ELK cluster in both primary and backup DC. So end goal of consumers
(Logstash) is to index logs in Elasticsearch and show them using Kibana. We
are replicating data in ELKs using mirror maker. It's not possible to
consume from both DCs at the same time as components which produ
Hello,
I just started learning Kafka and have the environment setup on my
hortonworks sandbox at home vmware.
test.csv is what I want the producer to send out:
more test1.csv ./kafka-console-producer.sh --broker-list
sandbox.hortonworks.com:6667 --topic kafka-topic2
1, abc
2, def
...
8, vwx
9,
If you create a topic with one partition they will be in order.
Alternatively if you publish with the same key for every message they will be
in the same order even if your topic has more than 1 partition.
Either way above will work for Kafka.
-hans
> On May 25, 2018, at 8:56 PM, Raymond Xie
10 matches
Mail list logo
