Re: Need to restart flink job on yarn as supervisord does

2018-01-02 Thread Shivam Sharma
; – Ufuk > > On Tue, Dec 26, 2017 at 9:30 AM, Shivam Sharma <28shivamsha...@gmail.com> > wrote: > > I am submitting my Flink Job on Yarn(Amazon EMR) > > > > On Tue, Dec 26, 2017 at 1:59 PM, Shivam Sharma <28shivamsha...@gmail.com > > > > wrote: > > &

Re: CEP: Dynamic Patterns

2017-12-26 Thread Shivam Sharma
t; > I found this ticket: https://issues.apache.org/jira/browse/FLINK-7129 and > was hoping you plan to add this feature soon :) > > This would make a powerful feature and open up many interesting use-cases. > > Meanwhile, can you suggest a way of implementing this use-case?

Re: Need to restart flink job on yarn as supervisord does

2017-12-26 Thread Shivam Sharma
​I am submitting my Flink Job on Yarn(Amazon EMR)​ On Tue, Dec 26, 2017 at 1:59 PM, Shivam Sharma <28shivamsha...@gmail.com> wrote: > Hi, > > My Flink job fails due to external activity like when Kafka goes down. I > want to restart my Flink job after certain time interval. &

Need to restart flink job on yarn as supervisord does

2017-12-26 Thread Shivam Sharma
Hi, My Flink job fails due to external activity like when Kafka goes down. I want to restart my Flink job after certain time interval. *I need to know best practices in this. How to restart Flink job automatically.* Thanks -- Shivam Sharma Data Engineer @ Goibibo Indian Institute

Re: Facing issue of long running Flink job on Yarn

2017-12-20 Thread Shivam Sharma
> data to Kafka: This server is not the leader for that topic-partition.“? Is > it possible that you had a network issue and the producer could not find > the leader broker? > > Best, > Stefan > > > Am 20.12.2017 um 10:57 schrieb Shivam Sharma <28shivamsha...@gmail.c

Facing issue of long running Flink job on Yarn

2017-12-20 Thread Shivam Sharma
ectors.kafka.FlinkKafkaProducer010 .invoke(FlinkKafkaProducer010.java:407) at org.apache.flink.streaming.api.operators.StreamSink.processElement(StreamSink. java:41) at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain. java:528) ... 52 more

Re: [jira] [Created] (FLINK-8256) Cannot use Scala functions to filter in 1.4 - java.lang.ClassCastException

2017-12-14 Thread Shivam Sharma
ions. > > > > > > That is a nice improvement, but it may have some impacts on tools that > > have > > > been build before. When you see classcast exceptions (like X cannot be > > cast > > > to X), that is probably caused by the fact that the classloader >

Re: [jira] [Created] (FLINK-8256) Cannot use Scala functions to filter in 1.4 - java.lang.ClassCastException

2017-12-13 Thread Shivam Sharma
0) > at org.apache.flink.util.InstantiationUtil.readObjectFromConfig( > InstantiationUtil.java:248) > at org.apache.flink.streaming.api.graph.StreamConfig. > getStreamOperator(StreamConfig.java:220) > > {code} > > However, replacing the functio

Re: Issues in migrating code from 1.3.2 to 1.4.0

2017-12-13 Thread Shivam Sharma
t 11:42 AM, Aljoscha Krettek <aljos...@apache.org> wrote: > Hi, > > Could you please provide a bit of context. From your second email I gather > that you're using the Table API, is that right? > > Best, > Aljoscha > > > On 14. Dec 2017, at 07:08, Shivam Sharma &l

Re: Issues in migrating code from 1.3.2 to 1.4.0

2017-12-13 Thread Shivam Sharma
Basically I am registering one scalarfunction in my code. Is there any change in writing or registering a User Defined function in flink 1.4.0? On Wed, Dec 13, 2017 at 11:55 PM, Shivam Sharma <28shivamsha...@gmail.com> wrote: > Hi > > I am getting following issues in working code

Issues in migrating code from 1.3.2 to 1.4.0

2017-12-13 Thread Shivam Sharma
) at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:248) at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:220) ... 10 more -- Shivam Sharma Data Engineer @ Goibibo Indian Institute Of Information Technology, Design and Manufacturing Jabalpur Mobile No- (+91) 8882114744

Re: Long running Flink job on yarn is getting failed after some time.

2017-12-12 Thread Shivam Sharma
$CopyingChainingOutput.pushToOperator(OperatorChain.java:528) ... 47 more My job is reading data from kafka and writing to another kafka topic. On Wed, Dec 13, 2017 at 12:54 PM, Shivam Sharma <28shivamsha...@gmail.com> wrote: > Hi, > > Flink version: 1.3.2 > > -- > Shivam Sharma &

Long running Flink job on yarn is getting failed after some time.

2017-12-12 Thread Shivam Sharma
Hi, Flink version: 1.3.2 -- Shivam Sharma Data Engineer @ Goibibo Indian Institute Of Information Technology, Design and Manufacturing Jabalpur Mobile No- (+91) 8882114744 Email:- 28shivamsha...@gmail.com LinkedIn:-*https://www.linkedin.com/in/28shivamsharma <https://www.linkedin.