Re: I'm back and ready to help grow our community!

2018-05-17 Thread Jesse Anderson
Congrats! On Thu, May 17, 2018, 6:44 PM Robert Burke wrote: > Congrats & welcome back! > > On Thu, May 17, 2018, 5:44 PM Huygaa Batsaikhan wrote: > >> Welcome back, Gris! Congratulations! >> >> On Thu, May 17, 2018 at 4:24 PM Robert Bradshaw >> wrote: >> >>> Congratulations, Gris! And welcome

Re: I'm back and ready to help grow our community!

2018-05-17 Thread Huygaa Batsaikhan
Welcome back, Gris! Congratulations! On Thu, May 17, 2018 at 4:24 PM Robert Bradshaw wrote: > Congratulations, Gris! And welcome back! > On Thu, May 17, 2018 at 3:30 PM Robin Qiu wrote: > > > Congratulations! Welcome back! > > > On Thu, May 17, 2018 at 3:23 PM Reuven Lax wrote: > > >> Congratu

Re: I'm back and ready to help grow our community!

2018-05-17 Thread Robert Bradshaw
Congratulations, Gris! And welcome back! On Thu, May 17, 2018 at 3:30 PM Robin Qiu wrote: > Congratulations! Welcome back! > On Thu, May 17, 2018 at 3:23 PM Reuven Lax wrote: >> Congratulations! Good to see you back! >> Reuven >> On Thu, May 17, 2018 at 2:24 PM Griselda Cuevas wrote: >>> H

Re: I'm back and ready to help grow our community!

2018-05-17 Thread Dmitry Demeshchuk
While this may be a bit off topic, I still want to say this. Congratulations on your graduation, Gris! On Thu, May 17, 2018 at 2:19 PM, Griselda Cuevas wrote: > Hi Everyone, > > > I was absent from the mailing list, slack channel and our Beam community > for the past six weeks, the reason was t

Re: I'm back and ready to help grow our community!

2018-05-17 Thread Jason Kuster
Wonderful Gris; warmest congratulations on the milestone and glad to have you back. :D On Thu, May 17, 2018 at 2:36 PM Kenneth Knowles wrote: > Congratulations!! > > On Thu, May 17, 2018 at 2:21 PM Griselda Cuevas wrote: > >> Hi Everyone, >> >> >> I was absent from the mailing list, slack chann

Re: I'm back and ready to help grow our community!

2018-05-17 Thread Kenneth Knowles
Congratulations!! On Thu, May 17, 2018 at 2:21 PM Griselda Cuevas wrote: > Hi Everyone, > > > I was absent from the mailing list, slack channel and our Beam community > for the past six weeks, the reason was that I took a leave to focus on > finishing my Masters Degree, which I finally did on Ma

Re: I'm back and ready to help grow our community!

2018-05-17 Thread Preston Marshall
Welcome back and congrats on the accomplishment! I hope to hear of some Beam meetups in NYC soon :) On Thu, May 17, 2018 at 5:23 PM Griselda Cuevas wrote: > Hi Everyone, > > > I was absent from the mailing list, slack channel and our Beam community > for the past six weeks, the reason was that I

Re: I'm back and ready to help grow our community!

2018-05-17 Thread Udi Meiri
Welcome back and congrats again! On Thu, May 17, 2018 at 2:23 PM Dmitry Demeshchuk wrote: > While this may be a bit off topic, I still want to say this. > > Congratulations on your graduation, Gris! > > On Thu, May 17, 2018 at 2:19 PM, Griselda Cuevas wrote: > >> Hi Everyone, >> >> >> I was abs

Fwd: Wait.on() - "Do this, then that" transform

2018-05-17 Thread Eugene Kirpichov
-- Forwarded message - From: Eugene Kirpichov Date: Mon, May 14, 2018 at 2:48 PM Subject: Wait.on() - "Do this, then that" transform To: dev , Mairbek Khadikov Hi folks, Wanted to give a heads up about the existence of a commonly requested feature and its first successful produ

Re: Controlling parallelism of a ParDo Transform while writing to DB

2018-05-17 Thread Jins George
I am a user running beam+flink. Flink runner currently exposes only the job level parallelism, not at an operator level. This is a really nice feature if can be supported. Flink's Datastream api provide that option though. Thanks, Jins George On 05/16/2018 10:24 PM, Chamikara Jayalath wrote

Re: java.io.IOException: Failed to close some writers

2018-05-17 Thread Eugene Kirpichov
Hmm I don't see anything obviously wrong with the code, assuming TableReference and TableSchema both implement a hashCode() consistent with equals() [I'm not 100% sure - could you check that?]... I'm hoping saving heap dumps will shed some light. On Thu, May 17, 2018 at 12:25 AM Jose Ignacio Honra

Re: Access to current watermark

2018-05-17 Thread Lukasz Cwik
This is not exposed programatically. Depending on which runner your using, you may be able to query for the watermark through its monitoring APIs if it is exposed as a metric and you know what it is called. This is likely to be brittle implementation and also the data is likely to be stale. On Thu

Re: Normal Spark Streaming vs Streaming on Beam with Spark Runner

2018-05-17 Thread Lukasz Cwik
On Wed, May 16, 2018 at 10:46 PM chandan prakash wrote: > Thanks Ismaël. > Your answers were quite useful for a novice user. > I guess this answer will help many like me. > > *Regarding your answer to point 2 :* > > > *"Checkpointing is supported, Kafka offset management (if I understand > whatyo

Access to current watermark

2018-05-17 Thread Jozef Vilcek
Hello, is there a way to observe `current watermark` when processing elements of triggered window? What I am trying to achieve is, to have on one window quite large late comer event allowance and I want to log information on how much late behind the watermark event did came in.

Re: java.io.IOException: Failed to close some writers

2018-05-17 Thread Jose Ignacio Honrado Benítez
Thanks Eugene, I will try "saveHeapDumpsToGcsPath" to see if there are any head dumps that I am missing. I am using DynamicDestinations, indeed. You can find the code here: https://pastebin.com/yrmXFQLH Note: DocumentWithSchema is a case class that contains the document JSON along with its Table