Re: Questions for platform to choose

2019-08-22 Thread Liam Clarke-Hutchinson
Hi Eliza, As I mentioned to you in the Kafka mailing list when you asked this there, there are pros and cons to all of the technologies you've mentioned, and you really need to sit down and try each solution to see what suits your needs best. Kind regards, Liam Clarke On Wed, Aug 21, 2019 at

Re: Questions for platform to choose

2019-08-21 Thread Magnus Nilsson
Well, you are posting on the Spark mailing list. Though for streaming I'd recommend Flink over Spark any day of the week. Flink was written as a streaming platform from the beginning quickly aligning the API with the theoretical framework of Google's Dataflow whitepaper. It's awesome for

Re: Questions for platform to choose

2019-08-21 Thread Mich Talebzadeh
Hi, What is the definition of real time here? The engineering definition of real time is roughly fast enough to be interactive. However, I put a stronger definition. In real time application or data, there is no such thing as an answer which is supposed to be late and correct. The timeliness is

Re: Questions for platform to choose

2019-08-21 Thread Eliza
Also I found a excellent article for comparision. https://www.linkedin.com/pulse/spark-streaming-vs-flink-storm-kafka-streams-samza-choose-prakash regards. on 2019/8/21 16:53, Eliza wrote: Hi, on 2019/8/21 16:44, Aziret Satybaldiev wrote: In my experience, Kafka + Spark streaming + (perhaps

Re: Questions for platform to choose

2019-08-21 Thread Eliza
Hi, on 2019/8/21 16:44, Aziret Satybaldiev wrote: In my experience, Kafka + Spark streaming + (perhaps HBase if you want to store the metrics) is so far the best combo. Not only because the technology is mature, but also because there are a lot of examples available on the web and books. They

Re: Questions for platform to choose

2019-08-21 Thread Aziret Satybaldiev
Hi Eliza, In my experience, Kafka + Spark streaming + (perhaps HBase if you want to store the metrics) is so far the best combo. Not only because the technology is mature, but also because there are a lot of examples available on the web and books. They typically should cover most of what you

Questions for platform to choose

2019-08-21 Thread Eliza
Hello, We have all of spark, flink, storm, kafka installed. For realtime streaming calculation, which one is the best above? Like other big players, the logs in our stack are huge. Thanks. - To unsubscribe e-mail: