Re: RocksDB error with flink 1.2.0

I ended up combining all the patterns into one giant CEP pattern and then filtering the output of that pattern instead. That way it was only one RocksDB instance which led to large checkpoints instead of lots of small checkpoints. This seems to work, but I still need to do more testing around wheth

RuntimeException: Error obtaining the sorted input: Thread 'SortMerger Reading Thread' terminated due to an exception: null

Hi everyone, what does mean that following exception, if I run my gelly program? Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed. at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$7.apply$mcV$sp

Flink with Hibernate ORM doesn't release memory after job is canceled

Hi Flink users, Does anyone use Flink with Hibernate ORM and have some examples? We have a job which reads a list of topics from Postgres accessed by Hibernate ORM as parameter for Kafka consumer. The Mapper also reads information from Postgres with Hibernate ORM, meaning, both master and worker

Re: Flink Graphire Reporter stops reporting via TCP if network issue

Hi Chesnay, Many thanks for your reply. At the end, we have decided to change the infrastructure a bit and use StatD instead. This way, we don't need a custom reporter and it works fine. Thanks! Bruno On Fri, 5 May 2017 at 13:20 Chesnay Schepler wrote: > Hello, > > for Graphite, Flink uses th

Re: High Availability on Yarn

Thanks for the update Aljoscha. @Till Rohrmann, Can you please chim in? Also, we currently have a long running EMR cluster where we create one flink cluster per job – can we just choose to install Zookeeper when creating the EMR cluster and use one Zookeeper instanc

Flink + Kafka + avro example

Hi to all Flink users, we've just published on our Okkam public repository an example of using Flink 1.2.1 + Kafka 0.10 to exchange Avro objects[1]. We hope this could be helpful for new Flink users willing to play with Flink streaming. [1] https://github.com/okkam-it/flink-examples/blob/master/s

Async I/O & async-http-client & Netty

Hi there, just wanted to let you guys know that I was playing around with Async I/O and async-http-client and there seems to be an incompatibility somewhere on Netty. I wanted to use an async function to enrich my data with results coming from Foursquare's API and it works fine when adding the belo

Re: Flink Graphire Reporter stops reporting via TCP if network issue

Hello, for Graphite, Flink uses the DropWizard metrics reporter. I don't know at the moment whether it supports any kind of reconnecting functionality. I'm not sure whether i understood you correctly; did you try upgrading the DropWizard metrics-core/metrics-graphite dependencies? If that d

Flink Graphire Reporter stops reporting via TCP if network issue

Hi, We are using the Graphite reporter from Flink 1.2.0 to send the metrics via TCP. Due to our network configuration we cannot use UDP at the moment. We have observed that if there is any problem with graphite our the network, basically, the TCP connection times out or something, the metrics rep

Re: Window Function on AllWindowed Stream - Combining Kafka Topics

I tried the timestamp field as a string datatype as well as a Date object. Getting same error in both the cases; Please find the POJO file: import java.text.DateFormat; import java.text.ParseException; import java.text.SimpleDateFormat; import java.util.Date; import java.util.HashMap; import

Re: Window Function on AllWindowed Stream - Combining Kafka Topics

What’s the KeySelector you’re using? To me, this indicates that the timestamp field is somehow changing after the original keying or in transit. Best. Aljoscha > On 4. May 2017, at 22:01, G.S.Vijay Raajaa wrote: > > I tried to reorder and the window function works fine. but then after > proces

yarn commit problem

Hi All！ if I commit job on yarn with -d will show exception Caused by: java.lang.IllegalStateException: No operators defined in streaming topology. Cannot execute. If I delete -d will ok Can someone tell me Why？ thanks！