Re: Permission problems

2016-03-04 Thread Jean-Baptiste Onofré
Hi Max, I gonna fix that for you. Regards JB On 03/04/2016 06:56 PM, Maximilian Michels wrote: Hi Beamers, While working on migrating the Beam web site to Git, I came across problems with the LDAP permissions. According to Infra, I'm not part of the incubator group [1]. Now that we want to m

Re: Permission problems

2016-03-04 Thread P. Taylor Goetz
Any of your mentors ought to be able to take care of this for you. When you need help from mentors, it sometimes helps to prepend “[MENTORS]” to the subject to get their attention. -Taylor > On Mar 4, 2016, at 4:17 PM, Davor Bonaci wrote: > > This is something that JB took care for us. I beli

Re: Permission problems

2016-03-04 Thread Davor Bonaci
This is something that JB took care for us. I believe only a member of the Incubator PMC can fix it [1]. I did a quick check for all committers -- Max, I think you are the only committer without proper access right now. JB, is this something you could take care of? [1] http://www.apache.org/dev/p

Re: [PROPOSAL] Beam recipes (integration new IOs & transforms)

2016-03-04 Thread Kevin Sookocheff
We have been using ParDo for now as well. And it works fine. Our use case revolves around batching data — a typical case is outputting data to a rate-limited API. We can do that already with existing primitives but it would be nice to have helper functions and an interface for that. Totally underst

Permission problems

2016-03-04 Thread Maximilian Michels
Hi Beamers, While working on migrating the Beam web site to Git, I came across problems with the LDAP permissions. According to Infra, I'm not part of the incubator group [1]. Now that we want to merge the Flink Runner [2], I think I'll be unable to merge any valuable changes to the repository. I

Re: [PROPOSAL] Beam recipes (integration new IOs & transforms)

2016-03-04 Thread Dan Halperin
Hi Kevin, Can you say more about your needs for an UnboundedSink API? Basically, we have found most Sinks are fairly simple and can be implemented with just ParDo; so the Unbounded Sink API has generally been dropping in priority relative to all the work we've got to do for general Beam-readiness

Re: [GSOC 2016] zeppelin interpreter for Beam.

2016-03-04 Thread Minudika Malshan
Thanks a lot Amit.. I will have a look at it :) Regards, Minudika Minudika Malshan Undergraduate Department of Computer Science and Engineering University of Moratuwa. On Fri, Mar 4, 2016 at 7:34 PM, Amit Sela wrote: > Not sure if that's what you were looking for, but check this out > > htt

Re: New beam website!

2016-03-04 Thread Jean-Baptiste Onofré
Hi Max, fair enough. Regards JB On 03/04/2016 03:52 PM, Maximilian Michels wrote: Hi JB, Good question. I didn't create the page but one problem about relative URLs is changing the root directory or switching to https. So instead of relative URLs which should use {{ site.baseurl }}/resource f

Re: [PROPOSAL] Beam recipes (integration new IOs & transforms)

2016-03-04 Thread Jean-Baptiste Onofré
Hi Kevin, it's part of the area where we are working for the IO, yes. Right now, I'm working on the unbounded source (Java), but next one is unbounded sink (we will discuss with Dan about that). Regards JB On 03/04/2016 03:26 PM, Kevin Sookocheff wrote: One thing we have been waiting on as

Re: New beam website!

2016-03-04 Thread Maximilian Michels
Hi JB, Good question. I didn't create the page but one problem about relative URLs is changing the root directory or switching to https. So instead of relative URLs which should use {{ site.baseurl }}/resource for all links. That way, we can simply change baseurl in the _config.yml and we're good

Re: [PROPOSAL] Beam recipes (integration new IOs & transforms)

2016-03-04 Thread Kevin Sookocheff
One thing we have been waiting on as Dataflow users is the finalization of the unbounded sink API. I'm not sure if this would fit into the scope for this example proposal though. On Fri, Mar 4, 2016 at 4:13 AM Jean-Baptiste Onofré wrote: > Hi guys, > > all good things have an end: I'm back from v

Re: [GSOC 2016] zeppelin interpreter for Beam.

2016-03-04 Thread Amit Sela
Not sure if that's what you were looking for, but check this out https://github.com/apache/incubator-beam/blob/master/sdk/src/main/java/com/google/cloud/dataflow/sdk/runners/RecordingPipelineVisitor.java It's intended for internal unit tests but it's an example for a PipelineVisitor that traverses

Re: [GSOC 2016] zeppelin interpreter for Beam.

2016-03-04 Thread Minudika Malshan
Actually topic should be changed to "Beam interpreter for Zeppelin" :) Thanks Minudika Malshan Undergraduate Department of Computer Science and Engineering University of Moratuwa. On Fri, Mar 4, 2016 at 6:23 PM, Minudika Malshan wrote: > Hi all, > > I am looking forward to participate in gso

[GSOC 2016] zeppelin interpreter for Beam.

2016-03-04 Thread Minudika Malshan
Hi all, I am looking forward to participate in gsoc 2016 with above mentioned project. Please support me to understand the implementation of following scenario. Suppose the user types the code written using data flow sdk commands in a text box at the front end. We need to get that code (let's say

Re: New beam website!

2016-03-04 Thread Jean-Baptiste Onofré
Hi Max, I just cloned your repo and built using jekyll. I just wonder why not using relative URL (for images and js location) instead of absolute ? It would allow us to directly open the website in a browser. WDYT ? Regards JB On 03/01/2016 01:59 PM, Maximilian Michels wrote: As a summary

Re: New beam website!

2016-03-04 Thread Jean-Baptiste Onofré
Hi Max, thanks for the update. I will push for you. Regards JB On 03/01/2016 01:59 PM, Maximilian Michels wrote: As a summary from https://issues.apache.org/jira/servicedesk/customer/portal/1/INFRA-11318 it would work as follows: We use the 'asf-site' branch of the repository. When we change

Re: Reference document for lateness concept in Beam

2016-03-04 Thread Jean-Baptiste Onofré
Hi Kenn, Very interesting document, especially around watermark. I'm still reading it carefully, and I will propose suggestions in the doc directly. Thanks ! Regards JB On 03/01/2016 05:50 PM, Kenneth Knowles wrote: Hi everyone, I have prepared a Beam version of our reference document for t

[PROPOSAL] Beam recipes (integration new IOs & transforms)

2016-03-04 Thread Jean-Baptiste Onofré
Hi guys, all good things have an end: I'm back from vacation ;) This morning, I'm resuming my work on new IOs and transforms, data integration oriented. Dan already provided great feedbacks on the JMS one, and I'm updating it to use "new" Bounded and Unbounded interfaces. Unfortunately, right