Re: Creating a custom operator

2016-05-09 14:56 GMT+02:00 Simone Robutti : > >- You wrote you'd like to "instantiate a H2O's node in every task > manager". This reads a bit like you want to start H2O in the TM's JVM , but > I would assume that a H2O node runs as a separate process. So should it be

Re: Creating a custom operator

>- You wrote you'd like to "instantiate a H2O's node in every task manager". This reads a bit like you want to start H2O in the TM's JVM , but I would assume that a H2O node runs as a separate process. So should it be started inside the TM JVM or as an external process next to each TM. Also, do

Re: Creating a custom operator

Hi Simone, sorry for the delayed answer. I have a few questions regarding your requirements and a some ideas that might be helpful (depending on the requirements). 1) Starting / stopping of H2O nodes from Flink - You wrote you'd like to "instantiate a H2O's node in every task manager". This

Re: Creating a custom operator

I'm not sure this is the right way to do it but we were exploring all the possibilities and this one is the more obvious. We also spent some time to study how to do it to achieve a better understanding of Flink's internals. What we want to do though is to integrate Flink with another distributed

Re: Creating a custom operator

Hi Simone, you are right, the interfaces you extend are not considered to be public, user-facing API. Adding custom operators to the DataSet API touches many parts of the system and is not straightforward. The DataStream API has better support for custom operators. Can you explain what kind of

Re: Creating a custom operator

Hello Fabian, we delved more moving from the input you gave us but a question arised. We always assumed that runtime operators were open for extension without modifying anything inside Flink but it looks like this is not the case and the documentation assumes that the developer is working to a

Creating a custom operator

Hello, I'm trying to create a custom operator to explore the internals of Flink. Actually the one I'm working on is rather similar to Union and I'm trying to mimick it for now. When I run my job though, this error arise: Exception in thread "main" java.lang.IllegalArgumentException: Unknown