liferoad commented on code in PR #31862: URL: https://github.com/apache/beam/pull/31862#discussion_r1675975198
########## website/www/site/content/en/documentation/ml/large-language-modeling.md: ########## @@ -122,3 +122,51 @@ A `ModelHandler` requires parameters like: * `device` – The device on which you wish to run the model. If device = GPU then a GPU device will be used if it is available. Otherwise, it will be CPU. * `inference_fn` - The inference function to use during RunInference. * `large_model` - (see `Memory Management` above). Whether to use memory minimization techniques to lower the memory footprint of your model. + +### Troubleshooting Large Models + +#### Pickling errors + +When sharing a model across processes with `large_model=True` or using a custom model handler, Beam sends the input and output data across a process boundary. +To do this, it uses a serialization method known as [pickling](https://docs.python.org/3/library/pickle.html). Review Comment: shall we issue one warning about this pickling error when large_model=True is set? We could link this guide. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
