AMOOOMA opened a new pull request, #37103:
URL: https://github.com/apache/beam/pull/37103
Renamed the original model manager to ModelHandlerManager which is more
aligned to its function
Added Model Manager as a util class that offers managed access to models,
the client can request models without having to worry about managing GPU OOMs.
Also added various tests that checks the functions of all classes.
Added optional functionality to spawn new process in multiprocessshared to
support running models in parallel
Changes should be safe because they are defaulted to False.
#### Classes
**`GPUMonitor`**
* **`start()`**: Begins background memory polling.
* **`stop()`**: Stops polling.
* **`reset_peak()`**: Resets peak usage tracking.
* **`get_stats() -> (current, peak, total)`**: Returns memory stats.
**`ResourceEstimator`**
* **`is_unknown(model_tag: str) -> bool`**: Checks if model needs profiling.
* **`get_estimate(model_tag: str, default_mb: float) -> float`**: Returns
memory cost.
* **`set_initial_estimate(model_tag: str, cost: float)`**: Manually sets
cost.
* **`add_observation(active_snapshot, peak_memory)`**: Updates cost model
via NNLS solver.
**`ModelManager`**
* **`acquire_model(tag: str, loader_func: Callable) -> Any`**: Gets model
instance (handles isolation/concurrency).
* **`release_model(tag: str, instance: Any)`**: Returns model to pool.
* **`force_reset()`**: Clears all models and caches.
* **`shutdown()`**: Cleans up resources.
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example:
`addresses #123`), if applicable. This will automatically add a link to the
pull request in the issue. If you would like the issue to automatically close
on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more
tips on [how to make review process
smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit
[https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more
information about GitHub Actions CI or the [workflows
README](https://github.com/apache/beam/blob/master/.github/workflows/README.md)
to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
