Akka tcp address on AWS EMR between job/task managers

2019-06-23 Thread Andy Hoang
Hi guys, I want to use some client lib that integrated with akka system Flink aslo already created akka system job manager and those task manager can use akka by its akka.tcp addess My architecture stucture currently is: EMR (flink 1.8) 1 core node 1 master node Flink job on Yarn I have on

RE: 回复:RE: EventTimeTrigger源码求帮忙解读,求各位大佬帮帮忙,不胜感激

2019-06-23 Thread Shi Quan
Hi, 你已经在阅读源码了,可以也看看InternalTimerServiceImpl和TimerHeapInternalTimer这两个类的实现,加深对这块的理解。 Best, Quan Shi From: 840124434 Sent: Monday, June 24, 2019 11:36 AM To: user-zh Subject: 回复:RE: EventTimeTrigger源码求帮忙解读,求各位大佬帮帮忙,不胜感激 你好: 感谢回复~

??????RE: EventTimeTrigger??????????????????????????????????????????

2019-06-23 Thread 840124434
?? ~ ??onelementwindow??,??window??maxTimeStampwindow -- -- ??: "Shi Quan"; : 2019??6??24??(??) 11:07 ??:

Re: Linkage Error RocksDB and flink-1.6.4

2019-06-23 Thread Shu Su
Hi Andrea Actually It’s caused by Flink’s ClassLoader. It’s because flink use parent Classloader to load jar first and then you use it in your user’s code, then user-code classloader will load it again so it raised the error. There are two solutions. 1. Add scope “provided” to maven

Re: Linkage Error RocksDB and flink-1.6.4

2019-06-23 Thread Yun Tang
Hi Andrea Since I have not written Scala for a while, I wonder why you need to instantiate your ColumnFamilyOptions, BlockBasedTableConfig and DBOptions on JM side. As far as I can see, you could instantiate your on your TM side like code: val rocksdbConfig = new OptionsFactory() {

flink????????

2019-06-23 Thread ????2008
??flink demo ??~~

RE: EventTimeTrigger源码求帮忙解读,求各位大佬帮帮忙,不胜感激

2019-06-23 Thread Shi Quan
Hi, 这里是我之前对Window、State、Watermark三者关系的一个总结https://blog.csdn.net/xianzhen376/article/details/90415350 同时你的问题引发了新的思考,因为之前并没有关注onWaterMark的处理。简单的过了下源码后,我这里抛砖引玉几点: 1. registerEventTimeTimer 并不是注册定时器,是一个很轻量的实现,实际上是一个Queue,记录以namespace和key为索引的window.maxTimestamp,此处的window表示Element event

Batch mode with Flink 1.8 unstable?

2019-06-23 Thread Ken Krugler
Hi all, I’ve been running a somewhat complex batch job (in EMR/YARN) with Flink 1.8.0, and it regularly fails, but for varying reasons. Has anyone else had stability with 1.8.0 in batch mode and non-trivial workflows? Thanks, — Ken 1. TimeoutException getting input splits The batch job

[ANNOUNCE] Weekly Community Update 2019/25

2019-06-23 Thread Konstantin Knauf
Dear community, this week's update contains a few pointers to upcoming changes in the development process, two recent proposals around metrics as well as smaller discussions on the dev mailing list. Still processing the backlog of topics & discussions by including a few of them each week. As

Linkage Error RocksDB and flink-1.6.4

2019-06-23 Thread Andrea Spina
Dear community, I am running a Flink Job backed by RocksDB, version 1.6.4 and scala 2.11. At the job Startp the following exception happens (it's recorded by the Job Manager). *Caused by: java.lang.LinkageError: loader constraint violation: loader (instance of

Unable to left join on windowed session

2019-06-23 Thread Tal Shargal
Hi, I'm trying to run time-windowed-left-join but the results are identical to those that are returned from the same query but using INNER JOIN. Means, the records from the "left" side that have no match are not included in the output stream. SELECT a.name, b.name FROM TEST_A a left join TEST_B b