Job failing during restore in different cluster

2018-09-28 Thread shashank734
Hi, I am trying to move my job from one cluster to another cluster using Savepoint. But It's failing while restoring on the new cluster. In error, it's still trying to connect from some URL of old cluster. I have checked all the properties and configuration. Is flink save the URL's while

Re: ElasticSearch Checkpointing taking too much time

2018-09-12 Thread shashank734
-- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: ElasticSearch Checkpointing taking too much time

2018-09-12 Thread shashank734
Hi, vino, I have tried bot HDFS and filesystem and other checkpoints completed successfully so access is not the issue. For debug mode, I have to restart the app. I'll check and let you know thanks -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: ElasticSearch Checkpointing taking too much time

2018-09-12 Thread shashank734
Hi Hequn, Actually there are no error logs and to turn on debug mode I have to restart the app, Actually, I am using around 25-30 operators all others are completing successfully in less time only elastic search sink is taking too much time. I am using around 6 Elastic search sinks all are

Re: ElasticSearch Checkpointing taking too much time

2018-09-11 Thread shashank734
Update: I am using parallelism 1 on this... is this issue? -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

ElasticSearch Checkpointing taking too much time

2018-09-11 Thread shashank734
I am using flink 1.5.3, In this i am using elastic search sink. In this checkpoints and savepoints are failing, I have already given 50 minutes timeouts. After looking into details only elastic search sink checkpoints are taking time 30-35 mins. But state size and buffer size is 0 in that. Don't

Sink Multiple Stream Elastic search

2018-08-02 Thread shashank734
Hello, I am using Elastic search5 Connector. Can I use same connection while sinking multiple streams to Elastic search? Currently, I think it creates a different transport connection for each sink. I think it's creating a lot of connections with the cluster. Cause I am sinking 5-6 streams.

Re: How to set fix JobId for my application.

2018-05-14 Thread shashank734
Thanks for suggestion now using Kafka for information sharing between apps. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: How to set fix JobId for my application.

2018-05-03 Thread shashank734
Thanks for the response, Actually I know that. The main thing I have to use that job id of app A in the queryable state in other application B. In that case, I don't want to redeploy application B whenever I change something in application A. -- Sent from:

How to set fix JobId for my application.

2018-05-03 Thread shashank734
How I can set Fixed JobId for my flink Job. Cause queryable client required Job ID. So whenever I'll update or redeploy my queryable state job than JobID will change and i have to change and redeploy in queryable client app. Is there any way I can fix the jobID or dynamically pass in the client

Re: FlinkML

2018-04-18 Thread shashank734
There are no active discussions or guide on that. But I found this example in the repo : https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/ml/IncrementalLearningSkeleton.java

Re: Fw: 1.4.2 - Unable to start YARN containers with more than 1 vCores per Task Manager

2018-04-03 Thread shashank734
CHeck in your Yarn configuration, Are you using DeafaultResourceCalculater which only uses memory while allocating resources. So you have to change it to DominantResourceCalculator. yarn.scheduler.capacity.resource-calculator