A question about Triggers

I want to augment a POJO in Trigger's onElement method, specifically supply the POJO with the watermark from the TriggerContext. The sequence of execution is this sequence 1. call to add() in the accumulator for the window and save the POJO reference in the Accumulator. 2. call to onElement on T

Re: context in OnElement of Trigger

I think I got the 0 wrong, I do however want to figure out, how much I can depend on this watermark to represent the progress of the window. On Sat, Dec 16, 2017 at 9:06 PM, Vishal Santoshi wrote: > What does TriggerContext.getWaterMark() represent. Is it the system level > watermark for the win

context in OnElement of Trigger

What does TriggerContext.getWaterMark() represent. Is it the system level watermark for the window operator ? When I pull it out I get a value of 0 a hole bunch of times ...

Re: Flink vs Spark streaming benchmark

Hi, In case you haven't seen it yet. Here's an analysis and response to Databricks' benchmark [1]. Best, Fabian [1] https://data-artisans.com/blog/curious-case-broken-benchmark-revisiting-apache-flink-vs-databricks-runtime 2017-11-13 11:44 GMT+01:00 G.S.Vijay Raajaa : > Hi Guys, > > I have bee

Re: flink eventTime, lateness, maxoutoforderness

Take a look at the section of Flink documentation titled "Event Time and Watermarks": https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/event_time.html#event-time-and-watermarks Also read the excellent series "Streaming 101" and "102", has useful animations depicting the flow of time

flink eventTime, lateness, maxoutoforderness

eventTime, lateness, maxoutoforderness are all about time. event Time is the water mark time on the record. lateness is record time or the real word time? maxoutoforderness is record time or the real word time? dataStream.keyBy(row -> (String)row.getField(0)) .window(TumblingEventTimeWind

No configuration found for key 'akka.version' after upgrade to Flink 1.4

Hi All, After 1.4 upgrade, our unit tests failed to start flink in local mode. How can we fix this failure ? 07:31:34,388 INFO org.apache.flink.streaming.api.environment.LocalStreamEnvironment - Running job on local embedded Flink mini cluster 07:31:34,537 INFO org.apache.flink.runtime.min

Re: docker-flink images and CI

I'd be very happy if we had that integrated into our release process, yes. I'll see how much we can automate there. > On 15. Dec 2017, at 14:16, Ufuk Celebi wrote: > > I agree with Patrick's (cc'd) comment from the linked issue. What I > understand from the linked issue is that Patrick will tak

转发：回复： org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: FlinkStreaming

- 转发邮件信息 - 发件人： 靖先生 发送日期：

Re: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: FlinkStreaming

confirm that whether your runnable jar ball contains this class: org.apache.flink.streaming.runtime.tasks.StreamTaskException . Unzip it and check ? > On 16 Dec 2017, at 16:19, Soheil Pourbafrani wrote: > > Hey, I have Flink code passing all its dependencies through > ExecutionEnvirnment obj

org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: FlinkStreaming

Hey, I have Flink code passing all its dependencies through ExecutionEnvirnment object. I run my code remotely on cluster and it errors: Exception in thread "main" org.apache.flink.client.program.ProgramInvocationException: The program execution failed: Job execution failed. at org.apache.flink.cl

how flink extracts timestamp from transformed elements?

Hi All, The timestamp assigner is for one type, normally for the type from the source, but after several operators, the element type would change and the elements would be aggregated, if I do timeWindow again, how flink extracts timestamp from elements? For example, the fold operators aggregate 10