Re: Questions re: ExecutionGraph & ResultPartitions for interactive use a la Spark

2016-01-09 Thread kovas boguta
Reposting in hopes relevant people have returned from vacation: I'm impressed with the Flink API, it seems simpler and more composable than what I've seen elsewhere. I'm trying to see how to achieve a more interactive, REPL-driven experience, similar to Spark. I'm consuming Flinkfrom Clojure. Fo

Re: Working with storm compatibility layer

2016-01-09 Thread Matthias J. Sax
Hi, I just double checked the Flink code and during translation from Storm to Flink declareOuputFields() is called twice. You are right that is does the same job twice, but that is actually not a problem. The Flink code is cleaner this way to I guess we will not change it. About lifecyle: If you

Re: Working with storm compatibility layer

2016-01-09 Thread Shinhyung Yang
Dear Matthias, Thank you for replying! that sounds weird and should not happen -- Spout.open() should get > called exactly once. That's what I thought too. I'm new to both Storm and Flink so it's quite complicated for me to handle both yet; would it be helpful for me if I know storm's lifecyle

Re: Working with storm compatibility layer

2016-01-09 Thread Matthias J. Sax
Hello Shinhyung, that sounds weird and should not happen -- Spout.open() should get called exactly once. I am not sure about multiple calls to declareOuputFields though -- if might be called multiple times -- would need to double check the code. However, the call to declareOuputFields should be i

Re: Machine Learning on Apache Fink

2016-01-09 Thread Vasiliki Kalavri
Hi Ashutosh, Flink has a Machine Learning library, Flink-ML. You can find more information and examples the documentation [1]. The code is currently in the flink-staging repository. There is also material on Slideshare that you might find interesting [2, 3, 4]. I hope this helps! -Vasia. [1]: ht

Re: Machine Learning on Apache Fink

2016-01-09 Thread Ashutosh Kumar
I see lot of study materials and even book available for ml on spark. Spark seems to be more mature for analytics related work as of now. Please correct me if I am wrong. As I have already built my collection and data pre processing layers on flink , I want to use Flink for analytics as well. Thank

Machine Learning on Apache Fink

2016-01-09 Thread Ashutosh Kumar
I am looking for some study material and examples on machine learning . Are classification , recommendation and clustering libraries available ? What is the timeline for Flink as backend for Mahout? I am building a meta data driven framework over Flink . While building data collection and transform