Re: Graduation!

Very nice :-) Good to see this happening! On Tue, Jan 10, 2017 at 11:58 PM, Tyler Akidau wrote: > Congrats and thanks to everyone who helped make this happen! :-D > > -Tyler > > On Tue, Jan 10, 2017 at 2:20 PM Kenneth Knowles > wrote: > > This is really exciting. It is such a privilege to be i

Re: Graduation!

Congratulations to the community. :) On Jan 11, 2017 3:37 PM, "Stephan Ewen" wrote: > Very nice :-) > > Good to see this happening! > > On Tue, Jan 10, 2017 at 11:58 PM, Tyler Akidau > > wrote: > > > Congrats and thanks to everyone who helped make this happen! :-D > > > > -Tyler > > > > On Tue,

Re: Graduation!

Congratulations to everyone on this important milestone. On 11/01/2017 11:52, Neelesh Salian wrote: > Congratulations to the community. :) > > On Jan 11, 2017 3:37 PM, "Stephan Ewen" wrote: > >> Very nice :-) >> >> Good to see this happening! >> >> On Tue, Jan 10, 2017 at 11:58 PM, Tyler Akidau

Re: splitIntoBundles vs. generateInitialSplits

Eugene, that makes a lot of sense to me. Do you think it's worth filing a Jira ticket? On Wed, Jan 11, 2017 at 2:58 AM Eugene Kirpichov wrote: I agree that the methods are named somewhat confusingly, and ideally would be named the same. Both of the names miss some aspect of the underlying conce

Re: splitIntoBundles vs. generateInitialSplits

Hi Eugene and Stas, Just back from couple of days off and jump on this discussion. I agree with Stas: it's worth to create a Jira about that. The only "semantic" difference is unbounded vs bounded source, but the behavior is the same. Regards JB On 01/11/2017 04:26 PM, Stas Levin wrote: Eu

Re: [jira] [Created] (BEAM-1261) State API should allow state to be managed in different windows

I was under the impression that user state was scoped to a ParDo and was not shareable across multiple ParDos. Wouldn't rewindowing require the usage of multiple ParDos and hence not allow for state to be shared? On Tue, Jan 10, 2017 at 10:51 PM, Robert Bradshaw < rober...@google.com.invalid> wrot

Re: [jira] [Created] (BEAM-1261) State API should allow state to be managed in different windows

On Wed, Jan 11, 2017 at 8:59 AM, Lukasz Cwik wrote: > I was under the impression that user state was scoped to a ParDo and was > not shareable across multiple ParDos. Wouldn't rewindowing require the > usage of multiple ParDos and hence not allow for state to be shared? No, you'd do something lik

Re: Graduation!

This is sweet. Congratulations to everyone involved in making this happen, and I'm excited to see where we go from here. On Wed, Jan 11, 2017 at 5:09 AM, amarouni wrote: > Congratulations to everyone on this important milestone. > > > On 11/01/2017 11:52, Neelesh Salian wrote: > > Congratulation

Re: [jira] [Created] (BEAM-1261) State API should allow state to be managed in different windows

On Wed, Jan 11, 2017 at 9:43 AM Robert Bradshaw wrote: > On Wed, Jan 11, 2017 at 8:59 AM, Lukasz Cwik > wrote: > > I was under the impression that user state was scoped to a ParDo and was > > not shareable across multiple ParDos. Wouldn't rewindowing require the > > usage of multiple ParDos and

Re: [jira] [Created] (BEAM-1261) State API should allow state to be managed in different windows

Bundle processing order is indeterminate, wouldn't accessing user state of a different window lead to indeterminate state information. This seems to be even weaker then what you get from side inputs that are triggered multiple times. On Wed, Jan 11, 2017 at 10:01 AM, Tyler Akidau wrote: > On Wed

Re: Jenkins build is still unstable: beam_PostCommit_Java_RunnableOnService_Dataflow #2011

We are taking a few minutes to consider roll forwards vs roll back the improvements to PAssert. On Wed, Jan 11, 2017 at 11:11 AM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See RunnableOnService_Dataflow/2011/> > >

Re: Beam ML

That's right; what other info do you think would be useful? On Tue, Jan 10, 2017 at 11:09 AM, Kam Kasravi wrote: > Thanks Andrew > I think more information about the DRM operations and how persistence > would be done at the runner level. It looks like HDFS or spark caching is > currently being u

Re: Beam ML

Thanks Andrew - Since you're quite familiar with how Mahout backends (flink, spark, h20) bind and enable DRM and the API Mahout/Samsara exposes - I think the end goal would be to surface a JAVA/Python API as well as outline a declarative syntax that the various runners can adhere to (perhaps via s

Re: Beam ML

Mahout would have CSR support in the upcoming release as well as hybrid GPU/CPU execution depending on the work load. This discussion is better moved to dev@mahout, since the Mahout project has long abstracted out Spark, Flink and H2O for distributed linear algebra; and we are now adding swappable

Re: [jira] [Created] (BEAM-1261) State API should allow state to be managed in different windows

I think WindowMappingFn (https://issues.apache.org/jira/browse/BEAM-260 / https://s.apache.org/beam-windowmappingfn-1-pager) is a good fit for this. There are details to shake out. One big thing it does not address well (because it is focused only on GC thresholds) is specifically which windows ne

Starter issue

Hi, I would like to get started with contributing and thought I'd start with this, if that is ok: https://issues.apache.org/jira/browse/BEAM-1056 Could somebody please assign it to me? Best regards, Tim

Re: Starter issue

Hi Tim What's your Jira id ? Thanks Regards JB⁣​ On Jan 12, 2017, 06:48, at 06:48, Tim Taschke wrote: >Hi, > >I would like to get started with contributing and thought I'd start >with this, if that is ok: >https://issues.apache.org/jira/browse/BEAM-1056 > >Could somebody please assign it to me?

Re: Starter issue

Welcome Tim -- it's great to have you join our community! I found your name in JIRA and assigned the issue to you. Thanks for your (future) contribution. On Wed, Jan 11, 2017 at 11:27 PM, Jean-Baptiste Onofré wrote: > Hi Tim > > What's your Jira id ? > > Thanks > Regards > JB⁣​ > > On Jan 12, 2