schizophrenicmaniac opened a new pull request, #40574: URL: https://github.com/apache/beam/pull/40574
When a KeyedModelHandler is built from a list of KeyModelMappings, load_model() returns a _ModelHandlerManager. That object then loads and caches the per-key models itself, using its own count-based LRU. With use_model_manager=True, ModelManager only sees that wrapper, so the real models skip its memory tracking and eviction, and KeyModelPathMapping side-input updates don't work either. This PR takes the validation route from the issue. RunInference now raises a ValueError at construction time when use_model_manager=True is used with a multi-model KeyedModelHandler, including when it's wrapped by with_preprocess_fn, with_postprocess_fn or with_no_batching. The error suggests max_models_per_worker_hint instead. Single-model KeyedModelHandlers are unaffected. I also documented use_model_manager and model_manager_args in the RunInference docstring and added a note to KeyedModelHandler. Added a test in base_test.py that covers the plain and wrapped cases, plus the combinations that should still be allowed. Fixes #40539 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
