Hi there,
I would like to automate some of my tasks using Apache Kafka. Previously i
used to do the same using Apache Airflow and which worked fine. But i want
to explore the same using Kafka whether this works better than Airflow or
not.
1) Kafka runs on Server A
2) Kafka searches for a file na
Hi there,
I would like to automate some of my tasks using Apache Kafka. Previously i
used to do the same using Apache Airflow and which worked fine. But i want
to explore the same using Kafka whether this works better than Airflow or
not.
1) Kafka runs on Server A
2) Kafka searches for a file na
I have 2 systems
1. System I - A web based interface based on Oracle DB and No REST API
support
2. System II - Supports rest API's which also has web based interface
.
When a record created or updated in either of the system I want
propagate the data to other system . Ca
You can't you only get a guarantee on the order for each partition, not
over partitions. Adding partitions will possible make it a lot worse, since
items with the same key wll land in other partitions. For example with two
partitions these will be about the hashes in each partitions:
partition-0: 0
I want to use Kafka for notifications of changes to data in a
dataservice/database. For each object that changes, a kafka message will be
sent. This is easy and we've got that working no problem.
Here is my use case : I want to be able to fire up a process that will
1) determine the current lo
Hello,
We have a number of different scenarios within our company that we are
considering Kafka for.
There is one case in particular that has caused debate. The relevant
characteristics are:
-
Very high throughput - 1000's of messages/second.
-
Very bursty traffic with unpredictable su
In addition to what Guozhang said -
1) Since you are looking into Camus, this is probably a question for the
Camus mailing list. I believe it does automatically detect new topics.
2) There is no priority and we intend to solve the traffic spike problems
through quotas. But usually in most cases, i
Hello Bhavesh,
1) If auto.create.topics.enable is turned on and consumer is subscribing to
a wildcard topic, then producers can just send to new topics on the fly
which can be then captured by the consumers.
2) For now we do not have priority mechanism, but we do have some initial
plans on quotas
We are planning to use Apache Kafka to replace Apache Fume for mostly as
log transport layer. Please see the attached image which is similar use
case ( and deployment architecture ) at Linkedin (according to
http://sites.computer.org/debull/A12june/pipeline.pdf ). I have
following questions:
