Re: [DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

Great! On Wed, Dec 14, 2016 at 1:30 AM Matthias J. Sax wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > I think it's worth to announce this via news list. :) > > On 12/13/16 7:32 AM, Robert Metzger wrote: > > The commun...@flink.apache.org

Re: checkpoint notifier not found?

Not sure how to go from here. How do I create a PR for this? $ git branch * doc-checkpoint-notify master $ git push origin master remote: Permission to apache/flink.git denied to abhishsi. fatal: unable to access 'https://github.com/apache/flink.git/': The requested URL returned error: 403

Re: How to retrieve values from yarn.taskmanager.env in a Job?

Till, Unfortunately, System.getenv() doesn't contain the expected variable even within the UDFs, but thanks for the info! In the Yarn setting, "the client" would be either: 1. the bin/flink executable (with configuration based on where it's run from… which might not be the same as the

PartitionedState and watermark of Window coGroup()

Hi Flink users, I'm a bit confused about how these two work when writing trigger for window coGroup(). Stream1.assignTimestampsAndWatermarks(new EventWatermark()) .coGroup(Stream2.assignTimestampsAndWatermarks(new EventWatermark())) .where(new

Re: [DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 I think it's worth to announce this via news list. :) On 12/13/16 7:32 AM, Robert Metzger wrote: > The commun...@flink.apache.org > has been created :) > > On Tue, Dec 13, 2016 at 10:43 AM, Robert Metzger >

Re: [DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

Exactly. On Tue, Dec 13, 2016 at 4:40 PM, Timur Shenkao wrote: > How to subscribe? > community-subscr...@flink.apache.org ? > > On Tue, Dec 13, 2016 at 6:32 PM, Robert Metzger > wrote: > >> The commun...@flink.apache.org has been created :) >> >> On

Re: In 1.2-SNAPSHOT, EventTimeSessionWindows are not firing untill the whole stream is processed

Hi Yassine, I managed to reproduce the problem. The cause is that we recently changed how the timer service is being cleaned up and now the watermark timers are not firing anymore. I'll keep you posted and hope to find a solution fast. Cheers, Aljoscha On Sun, 11 Dec 2016 at 22:10 Yassine

Standalone cluster layout

Hi folks, I am setting up a Flink cluster for testing, and I have a few questions regarding memory allocations: 1. Is there a recommended limit to the size of a TaskManager heap? I saw that Flink uses G1GC, so we can use dozens of GB. 2. Following the above question, should I use only

RE: Equivalent of Rx combineLatest() on a join?

Thanks Gábor, indeed it appears to work as expected. I found another way based on new evictors included in flink 1.2 (see FLINK-4174) that can remove elements anywhere in a window, for example based on element content. However the CoFlatMap solution you suggest is definitely simpler, I'm going

Re: Avro Parquet/Flink/Beam

Hi Billy, no, ParquetIO is in early stage and won't be included in 0.4.0-incubating (that I will prepare pretty soon). I will push the branch on my github (didn't have time yet, sorry about that). Regards JB On 12/13/2016 05:08 PM, Newport, Billy wrote: Is your parquetio going to be

RE: Avro Parquet/Flink/Beam

Is your parquetio going to be accepted in to 0.4? Also, do you have a link to your github? Thanks -Original Message- From: Jean-Baptiste Onofré [mailto:j...@nanthrax.net] Sent: Monday, December 12, 2016 11:49 AM To: user@flink.apache.org Subject: Re: Avro Parquet/Flink/Beam Hi Billy,

Re: [DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

How to subscribe? community-subscr...@flink.apache.org ? On Tue, Dec 13, 2016 at 6:32 PM, Robert Metzger wrote: > The commun...@flink.apache.org has been created :) > > On Tue, Dec 13, 2016 at 10:43 AM, Robert Metzger > wrote: > >> +1. I've requested

Re: [DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

The commun...@flink.apache.org has been created :) On Tue, Dec 13, 2016 at 10:43 AM, Robert Metzger wrote: > +1. I've requested the community@ mailing list from infra. > > On Tue, Dec 13, 2016 at 10:40 AM, Kostas Tzoumas > wrote: > >> It seems that

WindowFunction-extension, WindowedStream apply signature mismatch

(The following is a cross-post of a Stack Overflow question at https://stackoverflow.com/questions/41105142/windowfunction-extension-does-not-match-windowedstream-apply-signatures ) Why does the below error occur? Given this extension to WindowFunction: class TestWinFunc extends

Re: Bloom filter in Flink

Hi Gennady, this bloom filter is actually not distributed and only used internally as an optimization to reduce the amount of data spilled by a hash join. So, it is not meant to be user facing and not integrated in any API. You could of course use the code, but there might be better

Bloom filter in Flink

Hi Everyone, Noticed that Flink sources are contain distributed Bloom filter implementation, so wandering is somebody tried to use it in production for large scale

Re: How to retrieve values from yarn.taskmanager.env in a Job?

Hi Shannon, the job graph generation does not run in the task manager but on the client. The job graph is then submitted to the JobManager which then will deploy the individual tasks to the TaskManager. Thus, when generating the job graph the task manager environment variables are not accessible.

Re: [DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

+1. I've requested the community@ mailing list from infra. On Tue, Dec 13, 2016 at 10:40 AM, Kostas Tzoumas wrote: > It seems that several folks are excited about the idea - but there is still > a concern on whether this would be spam for the dev@ and user@ lists > (which >

Re: Equivalent of Rx combineLatest() on a join?

Dear Denis, I think you can do it with a simple CoFlatMapFunction (without windows): To use a CoFlatMapFunction, you need to first connect [1] your streams, which results in a ConnectedStreams. Then you can call flatMap on this, and give a CoFlatMapFunction to it (where two different callbacks