Re: [PROPOSAL] New sdk languages

2016-03-24 Thread Frances Perry
+Neville and Rafal for their take ;-) Excited to see this out. Multiple community driven SDKs are right in line with our goals for Beam. On Thu, Mar 24, 2016 at 3:04 PM, Ismaël Mejía wrote: > Addendum: actually the semantic model support is not so far away as I said > before (I havent finished

Re: [PROPOSAL] New sdk languages

2016-03-24 Thread Ismaël Mejía
Addendum: actually the semantic model support is not so far away as I said before (I havent finished reading and I thought they didn't support sessions), and looking at the git history the project is not so young either and it is quite active. On Thu, Mar 24, 2016 at 10:52 PM, Ismaël Mejía wrote:

Re: [PROPOSAL] New sdk languages

2016-03-24 Thread Ismaël Mejía
Hello, I just checked a bit the code and what they have done is interesting, the SCollection wrapper is worth a look, as well as the examples to get an idea of their intentions, the fact that the code looks so spark-lish (distributed collections like) is something that is quite interesting too:

Re: [PROPOSAL] New sdk languages

2016-03-24 Thread Jean-Baptiste Onofré
Hi Raghu, I agree: we should provide SDK in different languages, and DSLs for specific use cases. You got why I sent my proposal ;) Regards JB On 03/24/2016 07:14 PM, Raghu Angadi wrote: I would love to see Scala API properly supported. I didn't know about scio. Scala is such a natural fit

Re: getting started?

2016-03-24 Thread Jean-Baptiste Onofré
Hi Sri, you can start with samples, sharing your experience and use case with beam. You can also take a look on the Jira and submit PR. Thanks ! Regards JB On 03/24/2016 05:47 PM, Srikumar Chari wrote: Beam team, I am very excited about Beam, everything I’ve read about it is awesome. I just

Re: [PROPOSAL] New sdk languages

2016-03-24 Thread Raghu Angadi
I would love to see Scala API properly supported. I didn't know about scio. Scala is such a natural fit for Dataflow API. I am not sure of the policy w.r.t where such packages would live in Beam repo, but I personally would write my Dataflow applications in Scala. It is probably already the case b

getting started?

2016-03-24 Thread Srikumar Chari
Beam team, I am very excited about Beam, everything I’ve read about it is awesome. I just joined this alias and am interested in becoming a contributor. Was wondering if there a natural place for me to start? I’ve mostly trying to piece things from a scattering of all the docs/blogs. Sorry for

Re: [HEADS UP] Directory reorganization

2016-03-24 Thread Jean-Baptiste Onofré
Great, let me move forward on this. Thanks, Regards JB On 03/24/2016 05:19 PM, Davor Bonaci wrote: No doubt; that's the direction we are heading into. TextIO and AvroIO will stay in core; but GCP-related IOs like BigQuery go out. On Thu, Mar 24, 2016 at 12:01 AM, Jean-Baptiste Onofré wrote:

Re: [HEADS UP] Directory reorganization

2016-03-24 Thread Davor Bonaci
No doubt; that's the direction we are heading into. TextIO and AvroIO will stay in core; but GCP-related IOs like BigQuery go out. On Thu, Mar 24, 2016 at 12:01 AM, Jean-Baptiste Onofré wrote: > Hi Davor, > > It's a great move ! Thanks ! > > I propose to create the io module and first move "cont

[PROPOSAL] New sdk languages

2016-03-24 Thread Jean-Baptiste Onofré
Hi beamers, right now, Beam provides Java SDK. AFAIK, very soon, you should have the Python SDK ;) Spotify created a Scala API on top of Google Dataflow SDK: https://github.com/spotify/scio What do you think of asking if they want to donate this as Beam Scala SDK ? I planned to work on a Scal

Re: GitHub mirroring broken

2016-03-24 Thread Maximilian Michels
Yep, happens quite regularly actually. Usually resolved within a couple of hours. On Thu, Mar 24, 2016 at 7:43 AM, Jean-Baptiste Onofré wrote: > Yes, same issue with other Apache projects (including TLP). > > It's not yet completely fixed. > > Regards > JB > > > On 03/24/2016 06:32 AM, Davor Bona

Re: [HEADS UP] Directory reorganization

2016-03-24 Thread Jean-Baptiste Onofré
Hi Davor, It's a great move ! Thanks ! I propose to create the io module and first move "contrib" into io (and so remove contrib after). I'm not sure it makes sense to move the "core" IO (TextIO, etc) outside of the SDK, but for sure, new IOs (kafka, mqtt, JMS, etc) should come into io. Tho