Re: Result comparison from 2 DataStream Sources

2016-06-12 Thread iñaki williams
Hi Konstantin, and don't worry for your response. Your gist was perfect. Some days after my email I modify some things and I almost got it. Besides, as some other people recommended me, I have added a WindowStream because I am not looking for a global maximum, the value could increase or

Re: Result comparison from 2 DataStream Sources

2016-06-10 Thread Konstantin Knauf
Hi again, and again sorry for the late response. Regarding your first question: You can use a Key Selector Function [1]. Regarding your second question: If I understand your requirement correctly, this is already happening in my gist. By taking the union of both streams the local and away max

Re: Result comparison from 2 DataStream Sources

2016-06-02 Thread iñaki williams
Hi again! Thanks for your tips and gists, those are being really helpful. However, I probably didn't express my idea properly and it has been a litle misunderstood. I have been thinking about how to do this during these days and I will try to put a concrete example of what I want and if my way is

Re: Result comparison from 2 DataStream Sources

2016-05-29 Thread Konstantin Knauf
Hi again, from your diagram I have put together a gist, which I think does the job. I haven't had the time to test it though :( https://gist.github.com/knaufk/d1312503b99ee51554a70c9a22abe7e5 If you have any questions, let me know. It sometimes just takes a while until I answer ;) Cheers,

Re: Result comparison from 2 DataStream Sources

2016-05-28 Thread iñaki williams
Hello again! :) I have been checking the solution that you proprosed and but I don't really get how the KeyValueState helps on it. Could you please explain it a little bit more? I have drawn a diagram to make what I want clear, notice that the middle table doesn't need to be a table, it is just

Re: Result comparison from 2 DataStream Sources

2016-05-26 Thread iñaki williams
Hi again! Exactly, I am working on the Final Proyect of the carreer and I am using Apache Flink looking for surebets, only checking live tennis matches. Thanks for your tips, I will try it and I will notify with the outcomes. Thanks! 2016-05-26 20:33 GMT+02:00 Konstantin Knauf

Re: Result comparison from 2 DataStream Sources

2016-05-26 Thread Konstantin Knauf
Hi, interesting use case, you are looking for sure bets, I guess ;) Well, I think, what you want to then is probably to use a ConnectedStream, which you keyBy the "name" of both streams. The you can use CoFlatMap for comparison. You can use a KeyValueState zu save prices. In each map you can

Re: Result comparison from 2 DataStream Sources

2016-05-26 Thread iñaki williams
Hi! I will explain it with more details: I am comparing real time sport odds from two different betting Webpages. Assuming that I get just one java object (in reality I should get a List of in-play matches), for each DataStream and assuming that the name is the same of course, what I want to do

Re: Result comparison from 2 DataStream Sources

2016-05-26 Thread Konstantin Knauf
Hi, let me first check, if I understand your requirements correctly. I assume you want to compare attribute price for objects with the same name only, right? Further, I assume the objects are some kind of offer/bid with a timestamp? I think the solution heavily depends on how the records, which

Result comparison from 2 DataStream Sources

2016-05-26 Thread iñaki williams
Hi! I am working on something quite similar to the stockPrice example that is posted on the webpage ( https://flink.apache.org/news/2015/02/09/streaming-example.html) I am extracting some data from 2 different webpages and I represent the result using a java object. The diagram could be