A question about windowed values

2016-04-12 Thread Amit Sela
My instinct tells me that if a value does not belong to a specific window (in time) it's a part of a global window, but if so, what's the role of the "empty window". When should an element be a "value in an empty window" ?

TextIO.Read.Bound vs Create

2016-04-12 Thread Amit Sela
Why input values from *TextIO.Read.Bound *belong to an empty window while values from *Create* belong in a global window ? Thanks, Amit

Re: Massive package renaming coming

2016-04-12 Thread Jean-Baptiste Onofré
Hi Davor, +1 ! I already updated the PullRequest for annotations package. Thanks ! Regards JB On 04/13/2016 03:23 AM, Davor Bonaci wrote: We are preparing to do a massive, project-wide package rename from "com.google.cloud.dataflow" to "org.apache.beam". At the earliest, this could occur some

Massive package renaming coming

2016-04-12 Thread Davor Bonaci
We are preparing to do a massive, project-wide package rename from "com.google.cloud.dataflow" to "org.apache.beam". At the earliest, this could occur sometime tomorrow afternoon (Pacific time). Unfortunately, there's no way to do this without affecting ongoing work. We'll try to do it as quickly

Re: PROPOSAL: Apache Beam (virtual) meeting: 05/11/2016 08:00 - 11:00 Pacific time

2016-04-12 Thread Kenneth Knowles
Either works for me. Thanks James! On Tue, Apr 12, 2016 at 11:31 AM, Amit Sela wrote: > Anytime works for me. > > On Tue, Apr 12, 2016, 21:24 Jean-Baptiste Onofré wrote: > > > Hi James, > > > > 5/4 works for me ! > > > > Thanks, > > Regards > > JB > > > > On 04/12/2016 05:05 PM, James Malone wr

Re: PROPOSAL: Apache Beam (virtual) meeting: 05/11/2016 08:00 - 11:00 Pacific time

2016-04-12 Thread Amit Sela
Anytime works for me. On Tue, Apr 12, 2016, 21:24 Jean-Baptiste Onofré wrote: > Hi James, > > 5/4 works for me ! > > Thanks, > Regards > JB > > On 04/12/2016 05:05 PM, James Malone wrote: > > Hey JB, > > > > Sorry for the late reply! That is a good point; apologies I missed > noticing > > that c

Re: PROPOSAL: Apache Beam (virtual) meeting: 05/11/2016 08:00 - 11:00 Pacific time

2016-04-12 Thread Jean-Baptiste Onofré
Hi James, 5/4 works for me ! Thanks, Regards JB On 04/12/2016 05:05 PM, James Malone wrote: Hey JB, Sorry for the late reply! That is a good point; apologies I missed noticing that conflict. For everyone in the community, how would one of the following alternatives work? 5/4/2016 - 8:00 - 11

Re: PROPOSAL: Apache Beam (virtual) meeting: 05/11/2016 08:00 - 11:00 Pacific time

2016-04-12 Thread Lukasz Cwik
5/4/2016 works for me. On Tue, Apr 12, 2016 at 8:05 AM, James Malone < jamesmal...@google.com.invalid> wrote: > Hey JB, > > Sorry for the late reply! That is a good point; apologies I missed noticing > that conflict. For everyone in the community, how would one of the > following alternatives wor

Re: Add Beam up to https://analysis.apache.org?

2016-04-12 Thread Kenneth Knowles
Thanks for the hint! Filed https://issues.apache.org/jira/browse/INFRA-11642 . On Fri, Apr 8, 2016 at 9:08 AM, Nitin Lamba wrote: > Actually, I found out that one has to put in an infra request to hook-up to > ASF's Sonar. You could look at [1] for examples. > > HTH, > Nitin > [1] > > https://is

Re: Hive Runner

2016-04-12 Thread Jean-Baptiste Onofré
Actually, I started to work on it ;) Regards JB On 04/12/2016 06:43 PM, James Malone wrote: I do not believe anyone is actively working on a MR runner but we have a JIRA open to track interest and possibly track the development should work begin. https://issues.apache.org/jira/browse/BEAM-165

Re: Hive Runner

2016-04-12 Thread Jean-Baptiste Onofré
Hi, yes, I started the MapReduce runner. I can share where I am with you. Regards JB On 04/12/2016 06:19 PM, Zahoor Mohamed J wrote: That brings me to MapReduce runner... Any one working on that... Iam interested in helping and learning here.. Any pointers to start looking at To design

Re: Hive Runner

2016-04-12 Thread James Malone
I do not believe anyone is actively working on a MR runner but we have a JIRA open to track interest and possibly track the development should work begin. https://issues.apache.org/jira/browse/BEAM-165 On Tue, Apr 12, 2016 at 9:19 AM, Zahoor Mohamed J wrote: > That brings me to MapReduce runner

Re: Hive Runner

2016-04-12 Thread Zahoor Mohamed J
That brings me to MapReduce runner... Any one working on that... Iam interested in helping and learning here.. Any pointers to start looking at To design a MapReduce runner? ./Zahoor@iPhone > On 12-Apr-2016, at 7:42 PM, Jean-Baptiste Onofré wrote: > > We can imagine to translate some Fn

Re: PROPOSAL: Apache Beam (virtual) meeting: 05/11/2016 08:00 - 11:00 Pacific time

2016-04-12 Thread James Malone
Hey JB, Sorry for the late reply! That is a good point; apologies I missed noticing that conflict. For everyone in the community, how would one of the following alternatives work? 5/4/2016 - 8:00 - 11:00 AM Pacific time -or- 5/18/2016 - 8:00 - 11:00 AM Pacific time Best, James On Mon, Apr 11,

Re: Hive Runner

2016-04-12 Thread Jean-Baptiste Onofré
We can imagine to translate some Fn (DoPar) as Hive/SQL statements. I don't think it's super interesting, but why not. On the other hand, definitely, we will provide an IO for Hive. Regards JB On 04/12/2016 04:04 PM, Aljoscha Krettek wrote: Hi, what do you mean by Hive Runner? AFAIK Hive prov

Re: Hive Runner

2016-04-12 Thread Aljoscha Krettek
Hi, what do you mean by Hive Runner? AFAIK Hive provides an SQL like interface to data while execution is handled either by a MapReduce backend or the newer Tez backend. Therefore I don't think it makes sense to put Beam on Hive. Cheers, Aljoscha On Tue, 12 Apr 2016 at 15:38 Jean-Baptiste Onofré

Hive Runner

2016-04-12 Thread Ly, Kiet
I didn't see Hive runner in Beam. Is there a plan for Hive runner component? Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intend

Re: Hive Runner

2016-04-12 Thread Jean-Baptiste Onofré
Hi, you are right: for now, we have runners for spark, flink, google cloud platform. Some work are in progress to provide MapReduce, Gearpump runners. And we're also preparing a Runner API to simplify the way of writing runners. On the other hand, we will improve the website to provide a be