Re: Strata Conference this March 6-8

Maybe a good idea to try to organize a Beam meetup in london in the same dates in case some of the people around can jump in and talk too. On Wed, Jan 17, 2018 at 2:51 AM, Ron Gonzalez wrote: > Works for me... > > On Tuesday, January 16, 2018, 5:45:33 PM PST, Holden Karau > wrote: > > > How woul

Re: Some interesting use case

Yes you're right. I believe this is the use case that I'm after. So if I understand correctly, transforms that do aggregations just assume that the batch of data being aggregated is passed as part of a tensor column. Is it possible to hook up a lookup call to another Tensorflow Serving servable

Re: Some interesting use case

This sounds similar to the use case for tf.Transform, a library that depends on Beam: https://github.com/tensorflow/transform On Tue, Jan 16, 2018 at 5:51 PM Ron Gonzalez wrote: > Hi, > I was wondering if anyone has encountered or used Beam in the following > manner: > > 1. During machine le

Re: Strata Conference this March 6-8

Works for me... On Tuesday, January 16, 2018, 5:45:33 PM PST, Holden Karau wrote: How would folks feel about during the afternoon break (3:20-4:20) on the Wednesday (same day as Eugene's talk)? We could do the Philz which is a bit of a walk but gets us away from the big crowd and also

Some interesting use case

Hi,  I was wondering if anyone has encountered or used Beam in the following manner:   1. During machine learning training, use Beam to create the event table. The flow may consist of some joins, aggregations, row-based transformations, etc...  2. Once the model is created, deploy the model to s

Re: Strata Conference this March 6-8

How would folks feel about during the afternoon break (3:20-4:20) on the Wednesday (same day as Eugene's talk)? We could do the Philz which is a bit of a walk but gets us away from the big crowd and also lets folks not attending the conference but in the area join us. On Tue, Jan 16, 2018 at 5:29

Re: Strata Conference this March 6-8

Cool, let me know if you guys finally schedule it. I will definitely try to make it to Eugene's talk but having an informal BoF in the area would be nice... Thanks,Ron On Tuesday, January 16, 2018, 5:06:53 PM PST, Boris Lublinsky wrote: All for it Boris Lublinsky FDP Architect boris.lu

Re: Strata Conference this March 6-8

+1 to BoF On Tue, Jan 16, 2018 at 5:00 PM, Dmitry Demeshchuk wrote: > Probably won't be attending the conference, but totally down for a BoF. > > On Tue, Jan 16, 2018 at 4:58 PM, Holden Karau > wrote: > >> Do interested folks have any timing constraints around a BoF? >> >> On Tue, Jan 16, 2018

Re: Strata Conference this March 6-8

Probably won't be attending the conference, but totally down for a BoF. On Tue, Jan 16, 2018 at 4:58 PM, Holden Karau wrote: > Do interested folks have any timing constraints around a BoF? > > On Tue, Jan 16, 2018 at 4:30 PM, Jesse Anderson > wrote: > >> +1 to BoF. I don't know if any Beam talk

Re: Strata Conference this March 6-8

Do interested folks have any timing constraints around a BoF? On Tue, Jan 16, 2018 at 4:30 PM, Jesse Anderson wrote: > +1 to BoF. I don't know if any Beam talks will be on the schedule. > > > We could do an informal BoF at the Philz nearby or similar? > -- Twitter: https://twitter.com/holden

Strata Conference this March 6-8

+1 to BoF. I don't know if any Beam talks will be on the schedule. > We could do an informal BoF at the Philz nearby or similar?

Re: Strata Conference this March 6-8

We could do an informal BoF at the Philz nearby or similar? On Wed, Jan 17, 2018 at 11:23 AM Eugene Kirpichov wrote: > I'm giving a talk about splittable DoFn's > https://conferences.oreilly.com/strata/strata-ca/public/schedule/detail/63696?locale=zh > > There are no other talks with the word "B

Re: Strata Conference this March 6-8

I'm giving a talk about splittable DoFn's https://conferences.oreilly.com/strata/strata-ca/public/schedule/detail/63696?locale=zh There are no other talks with the word "Beam" in the title, unless I missed something. On Tue, Jan 16, 2018 at 4:11 PM Ron Gonzalez wrote: > Hi, > Will there be so

Strata Conference this March 6-8

Hi,  Will there be some talks or representation of Apache Beam at the coming Strata Conference this March 6-8?  Would be great to hear someone talk about how Beam's been used at their company as their core data integration platform. Thanks,Ron  

Re: KafkaIO reading from latest offset when pipeline fails on FlinkRunner

Is there a JIRA filed for this? I think this discussion should live in a ticket. Kenn On Wed, Jan 10, 2018 at 11:00 AM, Mingmin Xu wrote: > @Sushil, I have several jobs running on KafkaIO+FlinkRunner, hope my > experience can help you a bit. > > For short, `ENABLE_AUTO_COMMIT_CONFIG` doesn't me

Re: Switching to Java 8

Thanks. I think the one changing build is higher priority because it enables people to start modernizing the code (e.g. FileIO) and it'd be good to do that before 2.3 cut. I wasn't able to find the PR you mentioned in https://github.com/apache/beam/pulls , which one is it? On Mon, Jan 15, 2018 at

Re: [SQL] Windowing and triggering changes proposal

Thanks @Anton for the proposal. Window(w/ trigger) support in SQL is limited now, you're very welcome to join the improvement. There's a balance between injected DSL mode and CLI mode when we were implementing BealmSQL overall, not only widowing. Many default behaviors are introduced to make it wo

Re: bigquery issue

Look at the worker logs. This page shows how to log information and how to find what was logged: https://cloud.google.com/dataflow/pipelines/logging#cloud-dataflow-worker-log-example The worker logs contain a lot of information written by Dataflow and also by your code. Note that you may need to ch

Re: [SQL] Windowing and triggering changes proposal

I've commented on the doc. This is a really nice analysis and I think the proposal is good for making SQL work with Beam windowing and triggering in a way that will make sense to users. Kenn On Thu, Jan 11, 2018 at 4:05 PM, Anton Kedin wrote: > Hi, > > Wanted to gather feedback on changes I pro

Re: Jenkins build is still unstable: beam_PostCommit_Java_MavenInstall #5661

Thanks for the quick merge Ismaël. Build should be fine again now ;) Regards JB On 01/16/2018 06:27 PM, Ismaël Mejía wrote: We just merged a fix for this, it should be OK now. Sorry for the inconvience and thanks for the fix JB. On Tue, Jan 16, 2018 at 5:07 AM, Kenneth Knowles wrote: In cas

Re: Jenkins build is still unstable: beam_PostCommit_Java_MavenInstall #5661

We just merged a fix for this, it should be OK now. Sorry for the inconvience and thanks for the fix JB. On Tue, Jan 16, 2018 at 5:07 AM, Kenneth Knowles wrote: > In case anyone is following, this is > https://issues.apache.org/jira/browse/BEAM-3438 > > Kenn > > On Mon, Jan 15, 2018 at 5:08 PM, A

Re: [DISCUSS] State of the project

Thanks Davor for opening this discussion and HUGE +1 to do this every year or in cycles. I will fork this thread into a new one for the Culture / Project management part issues as suggested. About the diversity of users across runners subject I think that this requires more attention to unificatio

Re: [DISCUSS] State of the project: Community growth

Some ideas I have around the issues mentioned before: * Annual User Survey One important thing we should do is some sort of annual survey of users to have some feedback on the state of the Beam from the users point of view, we can take for example as a template the survey done by the Rust communi

Jenkins build is unstable: beam_Release_NightlySnapshot #654

See