Re: No key found restore States

Hi, right now, the way to do it is by using a custom operator, i.e. a OneInputStreamOperator. There you have the low-level control and can set timers based on watermarks or processing time. You can, for example look at StreamMap for a very simple operator or WindowOperator for an operator that does

Kafka producer sink message loss?

I am correct in assuming that the Kafka producer sink can lose message? I don't expect exactly-once semantics using Kafka as a sink given Kafka publishing guarantees, but I do expect at least once. I gather from reading the source that the producer is publishing messages asynchronously, as expect

Re: /newbie/ Share state between streams

Hi Andrey, State is scoped by operator instance so you cannot share state between two different operators even if they use the same name for the state -- "StateX" in your example. Also, you are correct that the recommended way of doing what you want is to use the connect() operations to connect t

Re: Event processing time with lateness

Super cool stuff On Fri, Jun 3, 2016 at 10:55 AM, Kostas Kloudas wrote: > You are welcome! > > > On Jun 3, 2016, at 4:40 PM, Igor Berman wrote: > > thanks Kosta > > On 3 June 2016 at 16:47, Kostas Kloudas > wrote: > >> Hi Igor, >> >> To handle late events in Flink you would have to implement y

/newbie/ Share state between streams

Hi, I am newbie in Flink and have questions about stream states. I can’t find answers in documentation, but if I just miss one, please link to doc) 1. Is ValueState (and other state classes) are ‘stream' scoped? So that, it is not possible to share same state with two (or more) different pipeli

Re: Event processing time with lateness

You are welcome! > On Jun 3, 2016, at 4:40 PM, Igor Berman wrote: > > thanks Kosta > > On 3 June 2016 at 16:47, Kostas Kloudas > wrote: > Hi Igor, > > To handle late events in Flink you would have to implement you own custom > trigger. > > To see a relati

Re: Event processing time with lateness

thanks Kosta On 3 June 2016 at 16:47, Kostas Kloudas wrote: > Hi Igor, > > To handle late events in Flink you would have to implement you own custom > trigger. > > To see a relatively more complex example of such a trigger and how to > implement it, > you can have a look at this implementation:

Re: Exception in thread main: No such exception errpr

Hello, Thank you for the advice. I have done that simulation and have successfully got correct results. But there is some problem which I have already raised before. My program basically reads an image file does facial detection (with the help of open cv library) and writes it back to disk. I hav

Re: Event processing time with lateness

Hi Igor, To handle late events in Flink you would have to implement you own custom trigger. To see a relatively more complex example of such a trigger and how to implement it, you can have a look at this implementation: https://github.com/dataArtisans/beam_comp/blob/master/src/main/java/com/da

Event processing time with lateness

Hi according to presentation of Tyler Akidau https://docs.google.com/presentation/d/13YZy2trPugC8Zr9M8_TfSApSCZBGUDZdzi-WUz95JJw/present Flink supports late arrivals for window processing, while I've seen several question in the userlist regarding late arrivals and answer was - sort of "not for al

Re: S3 as streaming source

You can check this docs page out for S3/AWS support: https://ci.apache.org/projects/flink/flink-docs-release-1.0/setup/aws.html On Fri, Jun 3, 2016 at 8:55 AM, Tzu-Li (Gordon) Tai wrote: > Hi, > > I've gave it a quick test and Chiwan is right. The methods `readFile`, > `readFileStream`, `readText

Re: S3 as streaming source

Hi, I've gave it a quick test and Chiwan is right. The methods `readFile`, `readFileStream`, `readTextFile` on StreamExecutionEnvironment works with the S3 scheme to stream from S3 objects. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/S3-