GitHub user lvfangmin opened a pull request:
https://github.com/apache/zookeeper/pull/590
[ZOOKEEPER-1177] Add the memory optimized watch manager for concentrate
watches scenario
The current HashSet based WatcherManager will consume more than 40GB memory
when
creating 300M watches.
This patch optimized the memory and time complexity for concentrate watches
scenario, compared to WatchManager, both the memory consumption and time
complexity improved a lot. I'll post more data later with micro benchmark
result.
Changed made compared to WatchManager:
* Only keep path to watches map
* Use BitSet to save the memory used to store watches
* Use ConcurrentHashMap and ReadWriteLock instead of synchronized to reduce
lock retention
* Lazily clean up the closed watchers
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lvfangmin/zookeeper ZOOKEEPER-1177
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zookeeper/pull/590.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #590
----
commit d4f996fdd760417c90ffb28fd63cc37dc87416c1
Author: Fangmin Lyu <allenlyu@...>
Date: 2018-08-06T21:43:22Z
add the memory optimized watch manager for concentrate watches
----
---
