Re: output writer

2015-09-09 Thread Fabian Hueske
icRegionType]) => > while(v.hasNext) out.collect(v.next)) > > > then in my custom output writer, write record checks whether the current > stream is the right one > if it is then write > otherwise close it and open a new one > > since they are grouped, only one stream per slot

Re: output writer

2015-09-09 Thread Michele Bertoni
pBy(0).reduceGroup((v, out : Collector[GenomicRegionType]) => while(v.hasNext) out.collect(v.next)) then in my custom output writer, write record checks whether the current stream is the right one if it is then write otherwise close it and open a new one since they are grouped, only one stream per slot

Re: output writer

2015-09-09 Thread Fabian Hueske
Radu Tudoran < >>>> radu.tudo...@huawei.com> ha scritto: >>>> >>>> Re-hi, >>>> >>>> I have double –checked and actually there is an OutputFormat interface >>>> in flink which can be extended. >>>> I believe t

Re: output writer

2015-09-08 Thread Michele Bertoni
July 30, 2015 11:34 AM To: user@flink.apache.org<mailto:user@flink.apache.org> Subject: Re: output writer Hi Michele, hi Radu Flink does not have such an OutputFormat, but I agree, it would be a valuable addition. Radu's approach looks like the way to go to implement this feature. @Rad

Re: output writer

2015-09-08 Thread Fabian Hueske
ng more outputformats I believe is something that >>> could be contributed. We should identify a couple of common formats. The >>> first one that comes in my mind is to have something for writing to memory >>> (e.g. memory buffer) >>> >>> >>> >>

Re: output writer

2015-09-08 Thread Michele Bertoni
anaging Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063, Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN From: Fabian Hueske [mailto:fhue...@gmail.com] Sent: Thursday, July 30, 2015 11:34 AM To: user@flink.apache.org<mail

Re: output writer

2015-09-08 Thread Fabian Hueske
09084330 >> Telephone: +49 891588344173 >> >> HUAWEI TECHNOLOGIES Duesseldorf GmbH >> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com >> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063, >> Managing Director: Jingwen TAO, Wanzhou MENG,

Re: output writer

2015-09-08 Thread Michele Bertoni
ursday, July 30, 2015 11:34 AM To: user@flink.apache.org<mailto:user@flink.apache.org> Subject: Re: output writer Hi Michele, hi Radu Flink does not have such an OutputFormat, but I agree, it would be a valuable addition. Radu's approach looks like the way to go to implement this feature. @Radu,

Re: output writer

2015-09-08 Thread Fabian Hueske
> Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN > > *From:* Fabian Hueske [mailto:fhue...@gmail.com ] > *Sent:* Thursday, July 30, 2015 11:34 AM > *To:* user@flink.apache.org > *Subject:* Re: output writer > > > Hi Michele, hi Radu > Flink does not have such an Ou

Re: output writer

2015-09-08 Thread Michele Bertoni
apache.org<mailto:user@flink.apache.org> Subject: Re: output writer Hi Michele, hi Radu Flink does not have such an OutputFormat, but I agree, it would be a valuable addition. Radu's approach looks like the way to go to implement this feature. @Radu, is there a way to contribute y

RE: output writer

2015-07-30 Thread Radu Tudoran
0, 2015 11:34 AM To: user@flink.apache.org Subject: Re: output writer Hi Michele, hi Radu Flink does not have such an OutputFormat, but I agree, it would be a valuable addition. Radu's approach looks like the way to go to implement this feature. @Radu, is there a way to contribute your Output

RE: output writer

2015-07-30 Thread Radu Tudoran
sellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063, Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN From: Fabian Hueske [mailto:fhue...@gmail.com] Sent: Thursday, July 30, 2015 11:34 AM To: user@flink.apache.org Subject: Re: output writer Hi Michele, hi Radu Flink does not ha

Re: output writer

2015-07-30 Thread Fabian Hueske
aging Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN > Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063, > Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN > > -Original Message- > From: Michele Bertoni [mailto:michele1.bert...@mail.polimi.it] >

RE: output writer

2015-07-30 Thread Radu Tudoran
r: Jingwen TAO, Wanzhou MENG, Lifang CHEN -Original Message- From: Michele Bertoni [mailto:michele1.bert...@mail.polimi.it] Sent: Thursday, July 30, 2015 10:15 AM To: user@flink.apache.org Subject: output writer Hi everybody, I have a question about the writer I have to save my dataset in d

output writer

2015-07-30 Thread Michele Bertoni
Hi everybody, I have a question about the writer I have to save my dataset in different files according to a field of the tuples let’s assume I have a groupId in the tuple, I need to store each group in a different file, with a custom name: any idea on how i can do that? thanks! Michele