Re: java.lang.Exception: TaskManager was lost/killed

Hi. I found the exception attached below, for our simple job. It states that our task-manager was killed du to exceed memory limit on 2.7GB. But when I look at Flink metricts just 30 sec before it use 1.3 GB heap and 712 MB Non-Heap around 2 GB. So something else are also using memory inside the c

Re: Flink 1.4.2 in Zeppelin Notebook

FYI: I finally managed to get the new Flink version running in Zeppelin. Besides adding the parameters mentioned below you have to build Zeppelin with profile scala-2.11 and the new Flink version 1.4.2. Best, Rico. Am 09.04.2018 um 14:43 schrieb Rico Bergmann: > > The error message is: > > org

Re: How to output a Table to Kafka?

Hi chi ma, A query with "group by" produces an updated table. Both UpsertStreamTableSink and RetractStreamTableSink can be used to emit an updated table. The main difference between UpsertStreamTableSink and RetractStreamTableSink is RetractStreamTableSink will encode an UPDATE change as a retract

Re: java.lang.Exception: TaskManager was lost/killed

Hi Chesnay, Don’t know if this helps, but I’d run into this as well, though I haven’t hooked up YourKit to analyze exactly what’s causing the memory problem. E.g. after about 3.5 hours running locally, it failed with memory issues. In the TaskManager logs, I start seeing exceptions in my code….

Re: java.lang.Exception: TaskManager was lost/killed

Same story here, 1.3.2 on K8s. Very hard to find reasons on why a TM is killed. Not likely caused by memory leak. If there is a logger I have turn on please let me know. On Mon, Apr 9, 2018, 13:41 Lasse Nedergaard wrote: > We see the same running 1.4.2 on Yarn hosted on Aws EMR cluster. The only

Re: java.lang.Exception: TaskManager was lost/killed

We see the same running 1.4.2 on Yarn hosted on Aws EMR cluster. The only thing I can find in the logs from are SIGTERM with the code 15 or -100. Today our simple job reading from Kinesis and writing to Cassandra was killed. The other day in another job I identified a map state.remove command to

Re: java.lang.Exception: TaskManager was lost/killed

We will need more information to offer any solution. The exception simply means that a TaskManager shut down, for which there are a myriad of possible explanations. Please have a look at the TaskManager logs, they may contain a hint as to why it shut down. On 09.04.2018 16:01, Javier Lopez w

Re: Flink Uploaded JAR Filename

Why don’t you parse the response from curl and use it to trigger the second request? That is easy automatable using Bash commands - or do I overlook something here? > On 9. Apr 2018, at 18:49, Pavel Ciorba wrote: > > Hi everyone > > I make 2 cURL POST requests to upload and run a Flink job. >

Flink Uploaded JAR Filename

Hi everyone I make 2 cURL POST requests to upload and run a Flink job. The issue is that I do it manually: First I do the upload, then using the returned name of the jar, I make a follow up run request. $ curl -X POST upload jar ... {status:success, filename: 12jhgjasd123asd12-ACTUALNAME.jar} $ c

Re: RocksDBMapState example?

Hi, Have you looked at the ctor call in : flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java around line 1261 ? Cheers

RocksDBMapState example?

Hi, I'd like to use RocksDB to store a key-value mapping table (with 45 million keys). Can someone please point me to an example of RocksDBMapState() constructor invocation? Or an explanation of constructor arguments?.. Thanks, Alex Nekrassov nekras...@att.com

Re: Re: java.lang.Exception: TaskManager was lost/killed

Hi, "are you moving the job jar to the ~/flink-1.4.2/lib path ? " -> Yes, to every node in the cluster. On 9 April 2018 at 15:37, miki haiat wrote: > Javier > "adding the jar file to the /lib path of every task manager" > are you moving the job jar to the* ~/flink-1.4.2/lib path* ? > > On

Re: Re: java.lang.Exception: TaskManager was lost/killed

Javier "adding the jar file to the /lib path of every task manager" are you moving the job jar to the* ~/flink-1.4.2/lib path* ? On Mon, Apr 9, 2018 at 12:23 PM, Javier Lopez wrote: > Hi, > > We had the same metaspace problem, it was solved by adding the jar file to > the /lib path of every ta

Re: Flink override config params (Docker)

Miki, So something like following will do the trick? *volumes:- /host/path/to/custom-flink-conf.yaml:/opt/flink/conf/flink-conf.yaml* 2018-04-09 14:11 GMT+03:00 miki haiat : > You can mount the conf folder and ovride the conf file. > > On Mon, 9 Apr 2018, 14:04 Pavel Ciorba, wrote: > >> Hi ev

Re: Flink 1.4.2 in Zeppelin Notebook

The error message is: org.apache.flink.client.program.ProgramInvocationException: The program execution failed: Communication with JobManager failed: Lost connection to the JobManager. Am 09.04.2018 um 14:12 schrieb kedar mhaswade: > Hmm. What error do you see on the Zeppelin console when you c

Re: Flink 1.4.2 in Zeppelin Notebook

Hmm. What error do you see on the Zeppelin console when you click the run (flink code) button after making these changes for flink interpreter config (I assume you restart the interpreter)? Regards, Kedar On Mon, Apr 9, 2018 at 12:50 AM, Rico Bergmann wrote: > Hi. > > Thanks for your reply. But

Re: Flink override config params (Docker)

You can mount the conf folder and ovride the conf file. On Mon, 9 Apr 2018, 14:04 Pavel Ciorba, wrote: > Hi everyone > > Is there a way to override the *conf/flink-conf.yaml* of the Flink Docker > container? > > I need to specify some params such as: > state.backend > state.backend.fs.checkpoint

Flink override config params (Docker)

Hi everyone Is there a way to override the *conf/flink-conf.yaml* of the Flink Docker container? I need to specify some params such as: state.backend state.backend.fs.checkpointdir state.checkpoints.dir etc. Thanks

Re: Re: java.lang.Exception: TaskManager was lost/killed

Hi, We had the same metaspace problem, it was solved by adding the jar file to the /lib path of every task manager, as explained here https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/debugging_classloading.html#avoiding-dynamic-classloading. As well we added these java option

Re: SSL config on Kubernetes - Dynamic IP

By the way Fabian, any chance this issue is looked into / the PR considered for 1.5? -- Christophe On Wed, Apr 4, 2018 at 2:41 PM, Fabian Hueske wrote: > Thank you Edward and Christophe! > > 2018-03-29 17:55 GMT+02:00 Edward Alexander Rojas Clavijo < > edward.roja...@gmail.com>: > >> Hi all, >>

Re: Re: java.lang.Exception: TaskManager was lost/killed

I've seen similar problem, but it was not a heap size, but Metaspace. It was caused by a job restarting in a loop. Looks like for each restart, Flink loads new instance of classes and very soon in runs out of metaspace. I've created a JIRA issue for this problem, but got no response from the devel

Re:Re: java.lang.Exception: TaskManager was lost/killed

thanks a lot,i will try it 在 2018-04-09 00:06:02，"TechnoMage" 写道： I have seen this when my task manager ran out of RAM. Increase the heap size. flink-conf.yaml: taskmanager.heap.mb jobmanager.heap.mb Michael On Apr 8, 2018, at 2:36 AM, 王凯 wrote: hi all, recently, i found a problem,it

Re: Flink 1.4.2 in Zeppelin Notebook

Hi. Thanks for your reply. But this also didn’t work for me. In the JM log I get an akka Error („dropping message for non-local recipient“). My setup: I have Flink running on Kubernetes cluster, version 1.4.2. zeppelin is version 0.8 using the flink interpreter compiled against flink 1.1.3.