damccorm commented on code in PR #23887: URL: https://github.com/apache/beam/pull/23887#discussion_r1009401772
########## website/www/site/content/en/documentation/ml/overview.md: ########## @@ -47,12 +47,26 @@ Further reading: ## Inference -There are several ways to use and deploy your model: -1. Making it available for online predictions via an API -2. Running it in real-time as new data becomes available in a pipeline -3. Running it in batch on an existing dataset +Beam provides different ways of implementing inference as part of your pipeline. This way you can run your ML model directly in your pipeline and apply it on big scale datasets, both in batch and streaming pipelines. + +### RunInference +The recommended way to implement inference is by using the [RunInference API](https://beam.apache.org/documentation/sdks/python-machine-learning/). RunInference takes advantage of existing Apache Beam concepts, such as the `BatchElements` transform and the `Shared` class, to enable you to use models in your pipelines to create transforms optimized for machine learning inferences. The ability to create arbitrarily complex workflow graphs also allows you to build multi-model pipelines. + +You can easily integrate your model in your pipeline by using the corresponding model handlers. A `ModelHandler` is an object that wraps the underlying model and allows you to configure its parameters. Model handlers are available for PyTorch, Scikit-learn and TensorFlow. Examples of how to use RunInference for PyTorch, Scikit-learn and TensorFlow are shown in this [notebook](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb). + +GPUs are optimized for training artificial intelligence and deep learning models as they can process multiple computations simultaneously. RunInference also allows you to use GPUs for significant inference speedup. An example of how to use RunInference with GPUs is demonstrated in a pipeline is demonstrated [here](/documentation/ml/runinference-metrics). Review Comment: ```suggestion GPUs are optimized for training artificial intelligence and deep learning models as they can process multiple computations simultaneously. RunInference also allows you to use GPUs for significant inference speedup. An example of how to use RunInference with GPUs is demonstrated [here](/documentation/ml/runinference-metrics). ``` ########## examples/notebooks/beam-ml/custom_remote_inference.ipynb: ########## @@ -0,0 +1,620 @@ +{ + "cells": [ + { + "cell_type": "code", + "source": [ + "# @title ###### Licensed to the Apache Software Foundation (ASF), Version 2.0 (the \"License\")\n", + "\n", + "# Licensed to the Apache Software Foundation (ASF) under one\n", + "# or more contributor license agreements. See the NOTICE file\n", + "# distributed with this work for additional information\n", + "# regarding copyright ownership. The ASF licenses this file\n", + "# to you under the Apache License, Version 2.0 (the\n", + "# \"License\"); you may not use this file except in compliance\n", + "# with the License. You may obtain a copy of the License at\n", + "#\n", + "# http://www.apache.org/licenses/LICENSE-2.0\n";, + "#\n", + "# Unless required by applicable law or agreed to in writing,\n", + "# software distributed under the License is distributed on an\n", + "# \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n", + "# KIND, either express or implied. See the License for the\n", + "# specific language governing permissions and limitations\n", + "# under the License" + ], + "metadata": { + "id": "paYiulysGrwR" + }, + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "# Remote inference in Beam\n", + "\n", + "The prefered way of running inference in Beam is by using the [RunInference API](https://beam.apache.org/documentation/sdks/python-machine-learning/). The RunInference API enables you to run your models as part of your pipeline in a way that is optimized for machine learning inference. It supports features such as batching, so that you do not need to take care of it yourself. For more info on the RunInference API you can check out the **RunInference notebook** (TODO link), which demonstrates how you can implement model inference in pytorch, scikit-learn and tensorflow.\n", Review Comment: Please add link instead of TODO -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
