Re: Beam Website Feedback

2023-02-06 Thread Reza Ardeshir Rokni
Hi Shlomi, If you intend to make use of GPU's for the purposes of Machine Learning Inference, the following resources may also be of interest to you: RunInference transform information: https://beam.apache.org/documentation/sdks/python-machine-learning/ You may also want to have a look at: https

Re: Beam Java starter project template

2022-02-15 Thread Reza Ardeshir Rokni
Hi, This is great! What do folks think about also having a less minimal set of starters? For Java I am thinking about protobuf / autovalue. For Python maybe an opinionated setup with tox etc... Again this would just contain 'hello' world samples to get folks going. Regards Reza On Wed, 9 Feb 20

Re: Unbounded pipeline: how to trigger a computation on empty windows

2021-01-19 Thread Reza Ardeshir Rokni
You may want to explore the use of Looping Timers. https://beam.apache.org/blog/looping-timers/ Please note that there are no order guarantees in OnProcess, some ideas on solutions to this are explored in: https://www.youtube.com/watch?v=Y_HoNNU6b3I Cheers On Wed, 20 Jan 2021 at 05:07, Reuven

Re: PTransform Annotations Proposal

2020-11-16 Thread Reza Ardeshir Rokni
+1 having a NeedsRam(x) annotation would be incredibly helpful. On Fri, 13 Nov 2020 at 05:57, Robert Burke wrote: > (Disclaimer, Mirac and their team did approach me about this beforehand as > their interest is in the Go SDK.) > > +1 I think it's a good idea. As you've pointed out, there are man

Re: Question about saving data to use across runner's instances

2020-11-16 Thread Reza Ardeshir Rokni
Hi, Do you have an upper bound on how large the file will become? If it's small enough to fit into a sideinput you may be able to make use of the Slow update sideinput pattern: https://beam.apache.org/documentation/patterns/side-inputs/ If not, then SatefulDoFn would be a good choice, but note a

Re: [Proposal] Add a new Beam example to ingest data from Kafka to Pub/Sub

2020-10-14 Thread Reza Ardeshir Rokni
Just a thought, but what if in the future there were templates for other runners? Then having a template folder would fit nicely no? We could even have a runner specific subfolder and maybe even a shared area for things that could be used by all templates for all runners? On Thu, 15 Oct 2020 at 1

Re: Stateful Pardo Question

2020-08-09 Thread Reza Ardeshir Rokni
+1 on having the behavior clearly documented, would also be great to try and add more stat and timer patterns to the Beam docs patterns page https://beam.apache.org/documentation/patterns/overview/. I think it might be worth thinking about describing these kind of patterns with an emphasis on the

Re: Stateful Pardo Question

2020-08-09 Thread Reza Ardeshir Rokni
ddy.com, wrote: > This is pretty much what the repro code does. The problem is that it > doesn’t work the way we would expect it should because the timer isn’t > called before processevent. > > > > *From: *Reza Ardeshir Rokni > *Reply-To: *"dev@beam.apache.org" > *

Re: Stateful Pardo Question

2020-08-07 Thread Reza Ardeshir Rokni
Hi, One possible approach ( have not tried it out, so might be missing cases..) but you can reset the timer from within the OnTimer code. So maybe you start the timer on the onprocess to go off at current+requiredGap. Then OnTimer, you check the list of elements and output a session if nothing ne

Re: [ANNOUNCE] New committer: Robin Qiu

2020-06-06 Thread Reza Ardeshir Rokni
Congratulations! On Wed, 20 May 2020 at 23:57, Austin Bennett wrote: > Congrats! > > On Tue, May 19, 2020, 8:32 PM Chamikara Jayalath > wrote: > >> Congrats Robin! >> >> On Tue, May 19, 2020 at 2:39 PM Rui Wang wrote: >> >>> Nice! Congrats! >>> >>> >>> >>> -Rui >>> >>> On Tue, May 19, 2020 at

Re: Non-trivial joins examples

2020-05-03 Thread Reza Ardeshir Rokni
A couple of things that are really nice here, 1- Domain specific (CTR in your example). We may find that eventually it's not possible / practical to build out generic joins for all situations. But with the primitives available in Beam and good 'patterns' domain specific joins could be added for di

Re: [docs] Python State & Timers

2019-04-24 Thread Reza Ardeshir Rokni
Pablo, Kenneth and I have a new blog ready for publication which covers how to create a "looping timer" it allows for default values to be created in a window when no incoming elements exists. We just need to clear a few bits before publication, but would be great to have that also include a python

Re: hi from DevRel land

2019-03-12 Thread Reza Ardeshir Rokni
at 7:21 AM Valentyn Tymofieiev >> wrote: >> >>> Hi Reza! >>> >>> Welcome to Beam. Very nice to have you onboard. Btw, the link seems >>> broken. >>> >>> Thanks, >>> Valentyn >>> >>> On Tue, Mar 12, 2019 a

hi from DevRel land

2019-03-12 Thread Reza Ardeshir Rokni
Hi Folks, Just wanted to say hi to the good folks in the Beam community in my new capacity as a Developer advocate for Beam/Dataflow @ Google. :-) At the moment I am working on a couple of blogs around the Timer and State API as well as some work on general patterns that I hope to contribute as d

Re: Another another new contributor! :)

2019-02-06 Thread Reza Ardeshir Rokni
Welcome! On Tue, 5 Feb 2019 at 23:34, Kenneth Knowles wrote: > Welcome Kyle! > > On Tue, Feb 5, 2019 at 4:34 AM Maximilian Michels wrote: > >> Welcome Kyle! Excited to see the Spark Runner moving towards portability! >> >> On 05.02.19 01:14, Connell O'Callaghan wrote: >> > Welcome Kyle! >> > >>

Re: New contributor: Michał Walenia

2019-01-31 Thread Reza Ardeshir Rokni
Welcome! On Thu, 31 Jan 2019 at 15:48, Michał Walenia wrote: > HI all, > thanks for a warm welcome :) > > Michał > > Wiadomość napisana przez Ahmet Altay w dniu > 30.01.2019, o godz. 21:32: > > Welcome Michał! > > On Wed, Jan 30, 2019 at 11:38 AM Kenneth Knowles wrote: > >> Welcome Michał! >>

Re: [ANNOUNCE] New committer announcement: Gleb Kanterov

2019-01-27 Thread Reza Ardeshir Rokni
Congratulations! On Sat, 26 Jan 2019 at 13:50, Pablo Estrada wrote: > Congrats:) > > On Fri, Jan 25, 2019, 9:24 PM Trần Thành Đạt wrote: > >> Congratulations Gleb! >> >> On Sat, Jan 26, 2019 at 4:19 AM David Morávek >> wrote: >> >>> Congratulations! >>> >>> Sent from my iPhone >>> >>> On 25 Ja

Re: [ANNOUNCE] New PMC member: Etienne Chauchot

2019-01-27 Thread Reza Ardeshir Rokni
Congratulations Etienne! On Sat, 26 Jan 2019 at 14:16, Ismaël Mejía wrote: > Congratulations Etienne! > > Le sam. 26 janv. 2019 à 06:42, Reuven Lax a écrit : > >> Welcome! >> >> On Fri, Jan 25, 2019 at 9:30 PM Pablo Estrada wrote: >> >>> Congrats Etienne :) >>> >>> On Fri, Jan 25, 2019, 9:24 P