Re: Class Cache

We have also noticed such behaviour when running on Yarn, had to restart the session for the changes in the jar to be picked up. On Mon, 31 Jul 2017 at 17:13, Ufuk Celebi wrote: > Hey Mike! > > Thanks for the detailed information about your setup. I'm also puzzled > by this...

[jira] [Created] (FLINK-7299) Write GenericRecord using AvroOutputFormat

Vishnu Viswanath created FLINK-7299: --- Summary: Write GenericRecord using AvroOutputFormat Key: FLINK-7299 URL: https://issues.apache.org/jira/browse/FLINK-7299 Project: Flink Issue Type

Re: Re: AVRO Union type support in Flink

icht > Betreff: Re: AVRO Union type support in Flink > Datum: Wed, 19 Jul 2017 10:26:24 -0400 > Von: Vishnu Viswanath <vishnu.viswanat...@gmail.com> > <vishnu.viswanat...@gmail.com> > An: Timo Walther <twal...@apache.org> <twal...@apache.org> &g

[jira] [Created] (FLINK-6726) Allow setting Timers in ProcessWindowFunction

vishnu viswanath created FLINK-6726: --- Summary: Allow setting Timers in ProcessWindowFunction Key: FLINK-6726 URL: https://issues.apache.org/jira/browse/FLINK-6726 Project: Flink Issue Type

[jira] [Created] (FLINK-6372) change-scala-version.sh does not change version for flink-gelly-examples

vishnu viswanath created FLINK-6372: --- Summary: change-scala-version.sh does not change version for flink-gelly-examples Key: FLINK-6372 URL: https://issues.apache.org/jira/browse/FLINK-6372 Project

Rescaling state

Hi All, Regarding the scale up and down feature mentioned in this talk: https://www.youtube.com/watch?v=IHSMnlWXkZ4=PLDX4T_cnKjD2UC6wJr_wRbIvtlMtkc-n2=18 Can this only be done when restarting the job from a save point? Is there a case where the state is transferred over from one node to another

Re: Incremental checkpoint branch

ewProfile.jspa? > name=xiaogang.shi> > > might > > be the right person to provide you its on-going status. > > > > cheers, > > Shaoxuan > > > > On Sat, Mar 4, 2017 at 5:14 AM, Vishnu Viswanath < > > vishnu.viswanat...@gmail.com> wrote: > &

Incremental checkpoint branch

Hi, Can someone point me to the branch where the ongoing work for incremental checkpoint is going on, I would like to try it out even if the work is not complete. I have a use case where the state size increase about ~1gb every 5 minutes. Thanks, Vishnu

Re: Flink survey by data Artisans

Works for me also. On Fri, Nov 18, 2016 at 12:35 PM, Stephan Ewen wrote: > Just checked it, the link works for me. > > > On Fri, Nov 18, 2016 at 7:20 PM, amir bahmanyari > wrote: > >> [image: Inline image] >> >> >> -- >>

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

mRecord is more of an internal class that should not really be user > facing, that's my motivation for replacing it. > > Cheers, > Aljoscha > > On Mon, 17 Oct 2016 at 19:23 Vishnu Viswanath < > vishnu.viswanat...@gmail.com> > wrote: > > > Hi Aljoscha, > > > >

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

T> to something like > TimestampedValue to not rely on tuples because they can be confusing for > people who write Scala code because they are not Scala tuples. That's not > strictly necessary, though, you can spin it however you like. > > Cheers, > Aljoscha > > On Fri, 7 Oct

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

not > sure if this functionality is captured in this interface. Basically, will > the typical remove() method from Iterators be available? > > Best regards, > > > -Original Message- > From: Vishnu Viswanath [mailto:vishnu.viswanat...@gmail.com] > Sent: Friday, Octobe

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

changes and also edit the FLIP. Regards, Vishnu On Wed, Oct 5, 2016 at 9:49 PM, Vishnu Viswanath < vishnu.viswanat...@gmail.com> wrote: > Thank you Aljoscha, > > Yes, I agree we don't need ProcessingTimeEvcitor. > I will change the current TimeEvictors to use EventTimeEvictor as &g

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

ow(...) > .evictor(new EventTimeEvictor()) > .apply(...) > > With this, we would just have to find a good way of passing the timestamps > in the Evictor interface and a good way of implementing the > EvictingWindowOperator. > > Cheers, > Aljoscha > > > On Sun, 18 Sep

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

ince I didn't come up with a better > solution yet. :-) > > Cheers, > Aljoscha > > > > On Sat, 30 Jul 2016 at 05:56 Vishnu Viswanath < > vishnu.viswanat...@gmail.com> > wrote: > > > Hi Aljoscha, > > > > 1. Regarding the Evictor inter

[SUGGESTION] Stack Overflow Documentation for Apache Flink

/documentation/apache-flink Regards, Vishnu Viswanath

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

che.org> wrote: > > > Hi, > > in fact, changing it to Iterable would simplify things because then > we > > would not have to duplicate code for the EvictingWindowOperator any more. > > It could be a very thin subclass of WindowOperator. > > > > Cheers, > >

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

s) { > > state.add(element) > > } > > > > This is very costly but the only way I see of doing this right now with > > every state backend. > > > > Cheers, > > Aljoscha > > > > On Mon, 25 Jul 2016 at 09:46 Radu Tudoran <radu.t

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

or partial disclosure, reproduction, > or dissemination) by persons other than the intended recipient(s) is > prohibited. If you receive this e-mail in error, please notify the sender > by phone or email immediately and delete it! > > > -Original Message- > From: Vishnu V

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

Hi, I have created a FLIP page for this enhancement https://cwiki.apache.org/confluence/display/FLINK/FLIP-4+%3A+Enhance+Window+Evictor Thanks, Vishnu On Thu, Jul 21, 2016 at 6:53 AM, Vishnu Viswanath < vishnu.viswanat...@gmail.com> wrote: > Thanks Aljoscha. > > On Thu, Jul 21,

Re: [DISCUSS] Enhance Window Evictor in Flink

> } > > after passing the elements to the window function. > > This is very inefficient but the only way I see of doing it right now. > > Cheers, > Aljoscha > > > On Thu, 21 Jul 2016 at 01:32 Vishnu Viswanath < > vishnu.viswanat...@gmail.com> > wrote: > >

Re: QueryableState, unable to register KvStateRegistryListener

uestions to the PR. > > How are you starting your cluster? My guess is that you are running > the cluster in local mode, which is not starting up the network > components. Is that the case? > > – Ufuk > > > On Thu, Jul 21, 2016 at 1:56 AM, Vishnu Viswanath > <vis

QueryableState, unable to register KvStateRegistryListener

ning false in the NetworkEnvironment.java. I am not sure how to proceed from here, do I need to set some configs for QueryableState to work? PS : I know this feature is not merged yet, but I was trying this out as part of my POC, any help is appreciated. Thanks, Vishnu Viswanath

Re: [DISCUSS] Enhance Window Evictor in Flink

() methods since it is an AppendingState, but that causes the evicted elements to come back when the trigger is fired next time. (It works fine when I use MemoryStateBackend) Is this expected behavior or am I missing something. Thanks, Vishnu On Mon, Jul 18, 2016 at 7:15 AM, Vishnu Viswanath

Re: [DISCUSS] Enhance Window Evictor in Flink

Hi Aljoscha, Thanks! Yes, I have the create page option now in wiki. Regards, Vishnu Viswanath, On Mon, Jul 18, 2016 at 6:34 AM, Aljoscha Krettek <aljos...@apache.org> wrote: > @Radu, addition of more window types and sorting should be part of another > design proposal. This is

Re: [DISCUSS] Enhance Window Evictor in Flink

t 10:22 PM, Vishnu Viswanath < vishnu.viswanat...@gmail.com> wrote: > Hi Aljoscha, > > I agree, the user will know exactly that they are creating an EventTime > based evictor or ProcessingTime based evictor looking at the code. > So do you think it will be ok to have multiple

Re: [DISCUSS] Enhance Window Evictor in Flink

behavior explicit at the call site is very > important, in my opinion. > > On Wed, 13 Jul 2016 at 16:28 Vishnu Viswanath < > vishnu.viswanat...@gmail.com> > wrote: > > > Hi, > > > > I was hoping to use the isEventTime method in the WindowAssigner to set > >

Re: [DISCUSS] Enhance Window Evictor in Flink

Hi, I was hoping to use the isEventTime method in the WindowAssigner to set that information in the EvictorContext. What do you think?. Thanks and Regards, Vishnu Viswanath, On Wed, Jul 13, 2016 at 10:09 AM, Aljoscha Krettek <aljos...@apache.org> wrote: > Hi, > I think the wa

Re: [DISCUSS] Enhance Window Evictor in Flink

to override the open and close in EvitingWindowOperator to make the reference of EvictorContext null. Thanks and Regards, Vishnu Viswanath, On Fri, Jul 8, 2016 at 7:40 PM, Vishnu Viswanath < vishnu.viswanat...@gmail.com> wrote: My thought process when asking if we can use state backend in

Re: [DISCUSS] Enhance Window Evictor in Flink

My thought process when asking if we can use state backend in window function was : can we add the elements to be evicted into some state and allow the evictAfter to read it from some context and remove it from the window? On Fri, Jul 8, 2016 at 7:30 PM, Vishnu Viswanath < vishnu.viswa

Re: [DISCUSS] Enhance Window Evictor in Flink

the internals of these work, so this might be a wrong question) Thanks and Regards, Vishnu Viswanath, On Fri, Jul 8, 2016 at 10:20 AM, Aljoscha Krettek <aljos...@apache.org> wrote: > Hi Vishnu, > how long would these patterns be? The Trigger would not have to sort the > element

Re: [DISCUSS] Enhance Window Evictor in Flink

sort the elements in the window everytime it sees an element. (I was planning to do this sorting in the window, which will be less often - only when the trigger fires) Thanks and Regards, Vishnu Viswanath, On Fri, Jul 8, 2016 at 6:04 AM, Aljoscha Krettek <aljos...@apache.org> wrote: Hi, >

Re: [DISCUSS] Enhance Window Evictor in Flink

for haven't arrived yet) wait and try again when the trigger gets fired next time. Thanks and Regards, Vishnu Viswanath, On Thu, Jul 7, 2016 at 9:19 AM, Radu Tudoran <radu.tudo...@huawei.com> wrote: > Hi, > > @Aljoscha - I can understand the reason why you are hesitant to intr

[jira] [Created] (FLINK-4174) Enhance Window Evictor

vishnu viswanath created FLINK-4174: --- Summary: Enhance Window Evictor Key: FLINK-4174 URL: https://issues.apache.org/jira/browse/FLINK-4174 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Enhance Window Evictor in Flink

be good to have the window sorted based on EventTime. Thanks and Regards, Vishnu Viswanath, On Wed, Jul 6, 2016 at 3:55 PM, Maxim <mfat...@gmail.com> wrote: > Actually for such evictor to be useful the window should be sorted by some > field, usually event time. What do you think

Re: [DISCUSS] Allowed Lateness in Flink

ut I would appreciate any suggestion on whether these are viable changes or will there any performance issue if these are done. Also any pointer on where to start(e.g, do I create a new class similar to EvictingWindowOperator that extends WindowOperator?) Thanks and Regards, Vishnu Viswanath, On Wed, J

Re: [DISCUSS] Releasing Flink 1.1.0

t; branch was forked off. > > Best, Fabian > > 2016-06-16 13:18 GMT+02:00 Vishnu Viswanath <vishnu.viswanat...@gmail.com > >: > > > Hi, > > > > Will queryable state available in 1.1.0? > > > > Thanks, > > Vishnu > > > > On Thursday,

Re: [DISCUSS] Releasing Flink 1.1.0

ed: > > >> - Resource manager refactoring > > >> > > >> Unmerged features: > > >> - Cassandra connector > > >> - Key groups ("rescaling from savepoints") > > >> - Queryable state > > >> > > >> I'm pretty sure I forgot many other features / pull requests, please > > post > > >> them to this thread. I'll collect them and create a Wiki page out of > it. > > >> > > >> Some immediate TODOs for us: > > >> - Which of the unmerged features are we going to add to the release? > > >> - Which blockers do we need to address before releasing? > > >> - Are there any volunteers for the release manager? > > >> > > >> > > >> Regards, > > >> Robert > > >> > > > -- Thanks and Regards, Vishnu Viswanath, *www.vishnuviswanath.com <http://www.vishnuviswanath.com>*