Re: Streaming model updates for the RunInference transform.

[Andy Ye via dev]

Left some comments. Thanks for this proposal Anand!

On Mon, Nov 21, 2022 at 4:22 PM Danny McCormick via dev 
wrote:

> Thanks for the writeup Anand, this looks like a good approach.
>
> > There is a *WIP
> *
>  section
> in the doc, where I am figuring out a better solution. I would love to hear
> any suggestions or alternatives for that section.
>
> Just wanted to boost this in case there are people who don't click through
> to the doc. The problem to solve is how to handle loading a new model
> without disrupting the current in progress threads that are performing
> inference on the old model (since loading a model can take minutes and take
> up a lot of space). Anand's current proposal is to load the second model
> into memory and require machines to have enough memory to store 2 models.
> If anyone has tried loading multiple large objects into a single process
> before, some insight on best practices could be helpful!
>
> Thanks,
> Danny
>
> On Mon, Nov 21, 2022 at 4:26 PM Anand Inguva via dev 
> wrote:
>
>> Hi,
>>
>> I created a doc
>> [1]
>> on a feature that I am working on for the RunInference
>> 
>> transform, where users can provide dynamic model updates via side inputs to
>> the RunInference transform.
>>
>> There is a *WIP
>> *
>> section in the doc, where I am figuring out a better solution. I would love
>> to hear any suggestions or alternatives for that section.
>>
>> Please go through the doc and let me know what you think.
>>
>> Thanks,
>> Anand
>>
>> [1]
>> https://docs.google.com/document/d/12j4bDwsIBhMN_8DNT2KGXPol7YS_G-DZFy6fjRsUGOQ/edit?usp=sharing
>>
>

Re: [ANNOUNCE] New committer: Yi Hu

[Andy Ye via dev]

Great job Yi!

On Fri, Nov 11, 2022 at 4:22 PM Ahmet Altay via dev 
wrote:

> Congratulations Yi!
>
> On Fri, Nov 11, 2022 at 1:57 PM Chamikara Jayalath via dev <
> dev@beam.apache.org> wrote:
>
>> Contrats Yi!
>>
>> On Thu, Nov 10, 2022 at 10:48 AM Kerry Donny-Clark via dev <
>> dev@beam.apache.org> wrote:
>>
>>> Great job Yi! I am happy to see your contributions recognized.
>>>
>>> On Thu, Nov 10, 2022 at 11:52 AM Yi Hu via dev 
>>> wrote:
>>>
 Thank you for the help of you all over the time, and I am glad to
 contribute and help with the community.

 Best,
 Yi

 On Thu, Nov 10, 2022 at 11:29 AM Alexey Romanenko <
 aromanenko@gmail.com> wrote:

> Congratulations! Well deserved!
>
> —
> Alexey
>
> On 9 Nov 2022, at 21:01, Tomo Suzuki via dev 
> wrote:
>
> Congratulations!
>
> On Wed, Nov 9, 2022 at 3:00 PM John Casey via dev 
> wrote:
>
>> Congrats! this is well deserved YI
>>
>> On Wed, Nov 9, 2022 at 2:58 PM Austin Bennett <
>> whatwouldausti...@gmail.com> wrote:
>>
>>> Congrats, and Thanks, Yi!
>>>
>>> On Wed, Nov 9, 2022 at 11:24 AM Valentyn Tymofieiev via dev <
>>> dev@beam.apache.org> wrote:
>>>
 I am with the Beam PMC on this, congratulations and very well
 deserved, Yi!

 On Wed, Nov 9, 2022 at 11:08 AM Byron Ellis via dev <
 dev@beam.apache.org> wrote:

> Congratulations!
>
> On Wed, Nov 9, 2022 at 11:00 AM Pablo Estrada via dev <
> dev@beam.apache.org> wrote:
>
>> +1 thanks Yi : D
>>
>> On Wed, Nov 9, 2022 at 10:47 AM Danny McCormick via dev <
>> dev@beam.apache.org> wrote:
>>
>>> Congrats Yi! I've really appreciated the ways you've
>>> consistently taken responsibility for improving our team's infra and
>>> working through sharp edges in the codebase that others have 
>>> ignored. This
>>> is definitely well deserved!
>>>
>>> Thanks,
>>> Danny
>>>
>>> On Wed, Nov 9, 2022 at 1:37 PM Anand Inguva via dev <
>>> dev@beam.apache.org> wrote:
>>>
 Congratulations Yi!

 On Wed, Nov 9, 2022 at 1:35 PM Ritesh Ghorse via dev <
 dev@beam.apache.org> wrote:

> Congratulations Yi!
>
> On Wed, Nov 9, 2022 at 1:34 PM Ahmed Abualsaud via dev <
> dev@beam.apache.org> wrote:
>
>> Congrats Yi!
>>
>> On Wed, Nov 9, 2022 at 1:33 PM Sachin Agarwal via dev <
>> dev@beam.apache.org> wrote:
>>
>>> Congratulations Yi!
>>>
>>> On Wed, Nov 9, 2022 at 10:32 AM Kenneth Knowles <
>>> k...@apache.org> wrote:
>>>
 Hi all,

 Please join me and the rest of the Beam PMC in welcoming a
 new committer: Yi Hu (y...@apache.org)

 Yi started contributing to Beam in early 2022. Yi's
 contributions are very diverse! I/Os, performance tests, 
 Jenkins, support
 for Schema logical types. Not only code but a very large 
 amount of code
 review. Yi is also noted for picking up smaller issues that 
 normally would
 be left on the backburner and filing issues that he finds 
 rather than
 ignoring them.

 Considering their contributions to the project over this
 timeframe, the Beam PMC trusts Yi with the responsibilities of 
 a Beam
 committer. [1]

 Thank you Yi! And we are looking to see more of your
 contributions!

 Kenn, on behalf of the Apache Beam PMC

 [1]

 https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

>>>
>
> --
> Regards,
> Tomo
>
>
>

Re: [ANNOUNCE] New committer: Ritesh Ghorse

[Andy Ye via dev]

Congrats Ritesh!

On Fri, Nov 4, 2022 at 9:26 AM Kerry Donny-Clark via dev <
dev@beam.apache.org> wrote:

> Congratulations Ritesh, I'm happy to see your hard work and community
> spirit recognized!
>
> On Fri, Nov 4, 2022 at 10:16 AM Jack McCluskey via dev <
> dev@beam.apache.org> wrote:
>
>> Congrats Ritesh!
>>
>> On Thu, Nov 3, 2022 at 10:12 PM Danny McCormick via dev <
>> dev@beam.apache.org> wrote:
>>
>>> Congrats Ritesh! This is definitely well deserved!
>>>
>>> On Thu, Nov 3, 2022 at 8:08 PM Robert Burke  wrote:
>>>
 Woohoo! Well done Ritesh! :D

 On Thu, Nov 3, 2022, 5:04 PM Anand Inguva via dev 
 wrote:

> Congratulations Ritesh.
>
> On Thu, Nov 3, 2022 at 7:51 PM Yi Hu via dev 
> wrote:
>
>> Congratulations Ritesh!
>>
>> On Thu, Nov 3, 2022 at 7:23 PM Byron Ellis via dev <
>> dev@beam.apache.org> wrote:
>>
>>> Congratulations!
>>>
>>> On Thu, Nov 3, 2022 at 4:21 PM Austin Bennett <
>>> whatwouldausti...@gmail.com> wrote:
>>>
 Congratulations, and Thanks @riteshgho...@apache.org!

 On Thu, Nov 3, 2022 at 4:17 PM Sachin Agarwal via dev <
 dev@beam.apache.org> wrote:

> Congrats Ritesh!
>
> On Thu, Nov 3, 2022 at 4:16 PM Kenneth Knowles 
> wrote:
>
>> Hi all,
>>
>> Please join me and the rest of the Beam PMC in welcoming a new
>> committer: Ritesh Ghorse (riteshgho...@apache.org)
>>
>> Ritesh started contributing to Beam in mid-2021 and has
>> contributed immensely to bringin the Go SDK to fruition, in addition 
>> to
>> contributions to Java and Python and release validation.
>>
>> Considering their contributions to the project over this
>> timeframe, the Beam PMC trusts Ritesh with the responsibilities of a 
>> Beam
>> committer. [1]
>>
>> Thank you Ritesh! And we are looking to see more of your
>> contributions!
>>
>> Kenn, on behalf of the Apache Beam PMC
>>
>> [1]
>>
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>
>

Re: Avoid breaking change when adding an additional parameter to the RunInference PTransform

[Andy Ye via dev]

Thanks Anand! I think following this framework will help developers create
ModelHandlers with fewer pitfalls, especially for specialized frameworks
like XGBoost that support a unique set of params for predict().

On Thu, Sep 29, 2022 at 11:16 AM Danny McCormick via dev <
dev@beam.apache.org> wrote:

> Thanks for the writeup Anand, I'm +1 on your recommendation in the doc:
> when we want to make parameterized changes that will propagate to the
> ModelHandler, I think we should modify the ModelHandler constructor rather
> than trying to propagate extra arguments through the run_inference function.
>
> Thanks,
> Danny
>
> On Thu, Sep 29, 2022 at 10:41 AM Anand Inguva via dev 
> wrote:
>
>> Hi,
>>
>> Recently I encountered a breaking change when adding an additional
>> parameter to the RunInference transform[1] and ModelHandler[2] in
>> pull/23266 .
>>
>> I explained what happened in a doc[3] and provided a few suggestions on
>> how to avoid this kind of behavior. Please take a look and let me know what
>> you think.
>>
>> Thanks,
>> Anand
>>
>> [1]
>> https://github.com/apache/beam/blob/17453e71a81ba774ab451ad141fc8c21ea8770c9/sdks/python/apache_beam/ml/inference/base.py#L291
>> [2]
>> https://github.com/apache/beam/blob/17453e71a81ba774ab451ad141fc8c21ea8770c9/sdks/python/apache_beam/ml/inference/base.py#L109
>> [3]
>> https://docs.google.com/document/d/1vJgGArgNTCkKo9BDxhf5jJtzr6yLTo1CRMw5B9ljT9o/edit?resourcekey=0-5TTi0_4oFPl0DM_g_04JBA#
>>
>

Re: Pass Custom namespaces to RunInference metrics

[Andy Ye via dev]

Thanks Anand for the great summary!

I'm leaning towards (solution 1) to allow users to pass in a custom
namespace for full user control. If there needs to be a certain grouping or
ordering of the metrics, the onus should be on the user to add an
appropriate grouping prefix such as 'RunInference_' in front of all the
custom namespaces.

Best,
Andy

On Mon, Sep 12, 2022 at 1:31 PM Anand Inguva via dev 
wrote:

> Hi all,
>
> I created a doc
> [1]
> to outline a few solutions on how to pass custom namespace to the
> RunInference[2] transform for better tracking and usage of metrics. Please
> go through it and let me know what you think. Any feedback would be
> appreciated.
>
> Thanks,
> Anand
>
> [1]
> https://docs.google.com/document/d/1qf5APv1kw5nbVZIMgqEp1N4XYcb0cJP0KZc9PrUYCO4/edit?usp=sharing
>
> [2]
> https://github.com/apache/beam/blob/2113ffcac3fa3d7522ceb22d03919e6edafe5e90/sdks/python/apache_beam/ml/inference/base.py#L269
>
>

Re: Benchmark tests for the Beam RunInference API

[Andy Ye via dev]

Appreciate the super clear summary of the different benchmark experiments!
This will add lots of value to potential users, especially when we
integrate GPU benchmarks. Thanks Anand!

Best,
Andy

On Thu, Aug 18, 2022 at 10:22 AM Danny McCormick via dev <
dev@beam.apache.org> wrote:

> I left a few comments, but overall this sounds like a good plan to me -
> thanks for the writeup!
>
> On Tue, Aug 16, 2022 at 9:36 AM Anand Inguva via dev 
> wrote:
>
>> Hi,
>>
>> I created a doc
>> [1]
>> which outlines the plan for the RunInference API[2] benchmark/performance
>> tests. I would appreciate feedback on the following,
>>
>>- Models used for the benchmark tests.
>>- Metrics calculated as part of the benchmark tests.
>>
>>
>> If you have any inputs or any suggestions on additional metrics/models
>> that would be helpful for the Beam ML community as part of the benchmark
>> tests, please let us know.
>>
>> [1]
>> https://docs.google.com/document/d/1xmh9D_904H-6X19Mi0-tDACwCCMvP4_MFA9QT0TOym8/edit#
>> [2]
>>  
>> https://github.com/apache/beam/blob/67cb87ecc2d01b88f8620ed6821bcf71376d9849/sdks/python/apache_beam/ml/inference/base.py#L269
>> 
>>
>>
>> Thanks,
>> Anand
>>
>

Re: Design Doc for Controlling Batching in RunInference

[Andy Ye via dev]

Thanks all for your feedback.

I've been looking into the batched DoFns, and will have a follow up on how
we can best interact with them.

On Mon, Aug 15, 2022 at 7:16 PM Robert Bradshaw  wrote:

> Thanks. I added some comments to the doc.
>
> I agree with Brian that it makes sense to figure out how this
> interacts with batched DoFns, as we'll want to migrate to that.
> (Perhaps they're already ready to migrate to as a first step?)
>
> On Fri, Aug 12, 2022 at 1:03 PM Brian Hulette via dev
>  wrote:
> >
> > Hi Andy,
> >
> > Thanks for writing this up! This seems like something that Batched DoFns
> could help with. Could we make a BatchConverter [1] that represents the
> necessary transformations here, and define RunInference as a Batched DoFn?
> Note that the Numpy BatchConverter already enables users to specify a batch
> dimension using a custom typehint, like NumpyArray[np.int64, (N, 10)] (the
> N identifies the batch dimension) [2]. I think we could do something
> similar, but with pytorch types. It's likely we'd need to define our own
> typehints though, I suspect pytorch typehints aren't already parameterized
> by size.
> >
> > Brian
> >
> >
> > [1]
> https://github.com/apache/beam/blob/master/sdks/python/apache_beam/typehints/batch.py
> > [2]
> https://github.com/apache/beam/blob/3173b503beaf30c4d32a4a39c709fd81e8161907/sdks/python/apache_beam/typehints/batch_test.py#L42
> >
> > On Fri, Aug 12, 2022 at 12:36 PM Andy Ye via dev 
> wrote:
> >>
> >> Hi everyone,
> >>
> >> I've written up a design doc [1] on controlling batching in
> RunInference. I'd appreciate any feedback. Thanks!
> >>
> >> Summary:
> >> Add a custom stacking function to RunInference to enable users to
> control how they want their data to be stacked. This addresses issues
> regarding data that have existing batching dimensions, or different sizes.
> >>
> >> Best,
> >> Andy
> >>
> >> [1]
> https://docs.google.com/document/d/1l40rOTOEqrQAkto3r_AYq8S_L06dDgoZu-4RLKAE6bo/edit#
>

Design Doc for Controlling Batching in RunInference

[Andy Ye via dev]

Hi everyone,

I've written up a design doc

[1]
on controlling batching in RunInference. I'd appreciate any feedback.
Thanks!

Summary:
Add a custom stacking function to RunInference to enable users to control
how they want their data to be stacked. This addresses issues regarding
data that have existing batching dimensions, or different sizes.

Best,
Andy

[1]
https://docs.google.com/document/d/1l40rOTOEqrQAkto3r_AYq8S_L06dDgoZu-4RLKAE6bo/edit#