Re: Flink solution to active - active Multi site cloud data ingestion

2019-05-17 Thread Melekh, Gregory
Thanks for your answer. We also thought about this solution, but finally we rejected it. In additional use case when needed to get unique reporting devices during time range this solution can't help us. Because could happen fallowing: East-Site reported devices: 1,2,3 West-Site reported devices:

Re: Flink solution to active - active Multi site cloud data ingestion

2019-05-15 Thread Seth Wiesman
In the future, these kinds of questions are more appropriate for the user mailing list (u...@flink.apache.org). Dev is for internal Flink development. On Wed, May 15, 2019 at 12:06 PM Seth Wiesman wrote: > Hi Gregory, > > The easiest solution would be to include the site in your key so that at

Re: Flink solution to active - active Multi site cloud data ingestion

2019-05-15 Thread Seth Wiesman
Hi Gregory, The easiest solution would be to include the site in your key so that at query time the rows from each site can be aggregated together. Instead of , the table would be and your query would become Select sum(value) FROM table GROUP BY key; Otherwise, you will need to get all that

Flink solution to active - active Multi site cloud data ingestion

2019-05-15 Thread Melekh, Gregory
Hello Flink Experts. We have Flink job consuming data from Kafka and ingest it to multi-site (Azure-east – Azure-west) replicated Cassandra. Now we have to aggregate data hourly. The problem is that device X can report once on site A and once on site B. This means that some messages for that