I had encountered similar problem and I was using stream to table join.
However if a record arrives late in the table, it would not join as per
stream to table join semantics.
Unfortunately windowing is not supported in stream to table join.
I suggest to use stream to stream join with appropriate
Hello,
According to the docs, Kafka Streams tasks pick from the partition with the
smallest timestamp to process the next record.
(
https://kafka.apache.org/documentation/streams/core-concepts#streams_out_of_ordering
)
One can also configure max.task.idle.ms so that Kafka Streams tasks wait
for al
I would recommend to try out Kafka's Streams API instead of Spark Streaming.
http://docs.confluent.io/current/streams/index.html
-Matthias
On 3/20/17 11:32 AM, Ali Akhtar wrote:
> Are you saying, that it should process all messages from topic 1, then
> topic 2, then topic 3, then 4?
>
> Or tha
Are you saying, that it should process all messages from topic 1, then
topic 2, then topic 3, then 4?
Or that they need to be processed exactly at the same time?
On Mon, Mar 20, 2017 at 10:05 PM, Manasa Danda
wrote:
> Hi,
>
> I am Manasa, currently working on a project that requires processing
Hi,
I am Manasa, currently working on a project that requires processing data
from multiple topics at the same time. I am looking for an advise on how to
approach this problem. Below is the use case.
We have 4 topics, with data coming in at a different rate in each topic,
but the messages in eac
