Re: Streaming API has a long delay at the beginning of the process.

Hi Fabian, Thanks a lot. I got a better understanding. > Operators are never GC'd (unless a job was cancelled) That's great information. Maybe, this is related to so called Managed Memory. The document will be better if detail documents about Memory Management exists. Thank you, Yuta On

Flink SocketTextStream source scheduled to a single machine

Hello! I'm trying to figure out how it happens: I'm having a program reading from multiple socketTextStream and these text streams feed into different data flow (and these data streams never connect in my job). It looks something similar to below: for(int i =0; i< hosts.length; i++) {

Re: Securing Flink Monitoring REST API

Unfortunately Flink does not yet support SSL mutual authentication nor any form of client authentication. There is an ongoing discussion about it: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-Service-Authorization-redux-td18890.html A workaround that I've seen is to

Re: Custom Serializers

One other thing :). Can i set tuple generic type dynamically? Meaning, build a tuple of N arity and build TupleSerializer based on those types. This because I'll only know these types based on user inputs. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Could not initialize keyed state backend.

Hi, that is not the case, and it also would not make too much sense if you think about restoring from a checkpoint in case of a machine failure. Is there a section in the Flink documentation that was confusing and has brought you to this assumption? Best, Stefan > Am 18.09.2017 um 15:56

Re: Could not initialize keyed state backend.

Hello, I thought that the checkpoints would be propagated to all the machines in the cluster when using a local filesystem. Thank you, Regards. - Best Regards, Pedro Chaves -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Clean GlobalWidnow state

I may be able to better know what is happening if I could get what is being stored in the state. Is there any way to read the RocksDB db state? Gerard -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Load distribution through the cluster

Hi, I'm experimenting a bit with the cluster. I didn't set any options about sharing slots and chains hoping that Flink decided autonomously how to balance the load through the nodes of the cluster. My cluster is composed by one job and task manager and two task manager. I noted that every time

Re: Custom Serializers

Ok, got it. Thanks -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Custom Serializers

you do need them, but only for the Event class. On 18.09.2017 13:38, nragon wrote: So, no need for typeinfo, comparator or factory? https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/types_serialization.html#defining-type-information-using-a-factory -- Sent from:

Re: Custom Serializers

So, no need for typeinfo, comparator or factory? https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/types_serialization.html#defining-type-information-using-a-factory -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Custom Serializers

If Parameters are always encapsulated in an Event, and the Event serializer knows how to deal with them, then you only need to implement a serializer etc. for the Event class. On 18.09.2017 13:20, nragon wrote: Sorry for bringing this up, any tips on this? -- Sent from:

Re: Custom Serializers

Sorry for bringing this up, any tips on this? -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Could not initialize keyed state backend.

Hi, are your checkpoints going against a local filesystem or against a distributed filesystem that is reachable from all task managers. This exception can happen in the first case: imagine your task restarts on a different machine, how could it find a file that was local to a different

Re: FlinkKafkaConsumer010 - Memory Issue

Hello, Sorry for the delay. The buffer memory of the Kafka consumer was piling up. Once I updated to the 1.3.2 version the problem no longer occurred. Pedro. - Best Regards, Pedro Chaves -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Taskmanager unable to rejoin job manager

Hi Marcus, thanks for reaching out with your problem. I'm not very experienced with the HA setup, but Till (in CC) might be able to help you. Best, Fabian 2017-09-14 16:57 GMT+02:00 Marcus Clendenin : > Hi all, > > > > I am having an issue where one of our task managers

Re: Securing Flink Monitoring REST API

Hi, sorry for the late response. Flink uses Netty for network communication which supports SSL client authentication. I haven't tried it myself, but would think that this should work in Flink as well if you configure the certificates correctly. We should update the docs to cover this aspect.

Re: Can't send data to another service in addSink

Please read the Basic API concepts guide in the documentation, in particular https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/api_concepts.html#lazy-evaluation. The short answer is that main() is called on the client, while the sink is executed on a taskmanager, i.e. in a

Re: ARIMA Model

Hi, no, such a model is not available. Best, Fabian 2017-09-17 20:01 GMT+02:00 Madhukar Thota : > is there a ARIMA(Autoregressive integrated moving average > ) > module available in Flink machine

Re: Get Job Id programmatically for SavePoint

Hi Rahul, probably is not the best way, but you could call jobmanager REST endpoint: http://[JOBMANAGER_HOST]:8081/joboverview who returns the list of running jobs and you could detect the right job id for example starting from the job name. I hope this could be helpfull. Mauro Il

Re: need instruction on how the Flink metric works

Hi, There are several possibilities: 1. Please check if reporter is set up ( guide ) For example, I would make sure my local JMXReporter service is up and running by